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SUMMARY 


The  ability  to  accurately  and  objectively  assess  an  individual's 
level  of  performance  on  the  job  is  important  to  Air  Force  systems  for 
personnel  selection,  assignment,  training  and  utilization.  Currently 
the  On-the-Job  Training  (OJT)  supervisor  is  responsible  for  the 
evaluation  of  an  individual  trainee's  task  proficiency.  Concern  has 
been  expressed  within  the  Air  Force  training  community  regarding  the 
variability  which  exists  among  supervisors  in  the  evaluation  of  OJT 
task  proficiency.  In  particular,  there  exists  a  lack  of 
standardization  with  regard  to  the  development  of  assessment 
instruments,  administration  of  performance  evaluations,  the  scoring  of 
results  and  providing  feedback. 

The  Task  Evaluation  Form  (TEF)  development  procedures  were 
developed  and  evaluated  by  Applied  Science  Associates,  Inc.  under 
contract  with  the  Air  Force  Human  Resource  Lab/ID  (Contract  No. 
F33615-82-C-0004).  The  TEF  development  procedures  allow  Subject-Matter 
Experts  (SMEs),  without  experience  or  training  in  assessment 
methodology,  to  develop  instruments  which  can  be  used  to  assess  OJT 
task  proficiency.  SMEs  apply  the  Task  Evaluation  Form  development 
procedures  to  a  specific  task  in  order  to  generate  a  TEF.  Evaluators 
then  use  the  TEFs  to  conduct  evaluations  of  OJT  task  performance. 
Potential  performer  actions  and  outcomes  are  divided  into  evaluation 
areas  (Time/Speed,  End  Product  Result,  Sequence-Following,  Safety,  and 
Tools,  Equipment,  and  Material  Use).  The  TEF  development  procedures 
guide  the  developer  in  determining  which  evaluation  areas  are  critical 
to  successful  task  performance.  The  developer  considers  every  step  or 
event  in  the  task  to  determine  at  which  points  an  evaluation  area 
should  be  evaluated.  Once  critical  task  events  or  steps  have  been 
identified  for  evaluation,  the  developer  describes  exactly  what 
observable  performer  actions  and  outcomes  reflect  successful  task 
performance.  These  descriptions  are  considered  performance  standards 
and  are  entered  on  the  TEF.  In  addition  to  guiding  the  developer  in 
identifying  and  describing  critical  aspects  of  task  performance,  the 
TEF  development  procedures  also  provide  instructions  for  creating  an 
evaluation  scenario  and  developing  a  chart  for  scoring  task 
performance.  A  task  proficiency  assessment  instrument  resulting  from 
application  of  the  TEF  development  procedures  reduces  evaluator 
subjectivity  with  regard  to:  setting  up  the  evaluation,  selecting 
aspects  of  the  task  for  evaluation,  making  decisions  about  acceptable 
outcomes  and  behaviors,  and  scoring  the  evaluator. 


The  instructions  contained  in  this  document  are  applicable  to  the 
development  of  TEFs  for  maintenance  or  equipment-oriented  tasks. 
Instructions  for  the  development  of  TEFs  for  non-equipment-oriented 
tasks  are  provided  in  another  document:  Task  Evaluation  Form: 


Development  Procedures  For  Non-Equipment-Oriented  Tasks. 
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PREFACE 


This  technical  paper  was  prepared  by  Applied  Science  Associates, 
Inc.  (ASA),  Valencia,  Pennsylvania,  under  Air  Force  Contract  Number 
F33615-82-C-0004.  Ms.  Ronnie  E.  Warm  was  the  Project  Scientist  and 
Director.  The  project  was  sponsored  by  the  Air  Force  Human  Resources 
Laboratory/Training  Systems  Division,  Lowry  AFB.  The  Contract  Monitor 
was  Mr.  Gerald  S.  Walker.  Captain  Richard  Dineen  was  the  Technical 
Contract  Monitor  until  his  retirement  in  July  1984,  at  which  time  Major 
Martin  Costellic  became  the  Technical  Contract  Monitor. 

This  study  is  one  of  a  series  of  related  studies  under  the 
Program;  Systems  Integration,  Transition,  and  Technical  Support.  The 
objective  of  this  program  is  to  provide  support  for  the  Advanced 
On-the-Job  Training  System  (ACTS).  Task  proficiency  assessment 
instruments  are  necessary  for  assessing  training  on  an  individual  and 
unit  level  within  the  AOTS.  However,  techniques  for  performing 
reliable,  valid,  and  standardized  task  proficiency  evaluations  do  not 
exist  currently.  This  document  is  one  major  result  of  the  development 
phase  of  an  exploratory  development  study  that  had  the  purpose  of 
deriving  methodology  for  the  development  of  OJT  task  proficiency 
assessment  instruments.  The  procedures  contained  in  this  document 
guide  Subject-Matter  Experts  (SMEs)  in  the  construction  of  task 
proficiency  assessment  instruments.  The  instruments  (Task  Evaluation 
Forms)  resulting  from  the  application  of  the  procedures  can  be  used  by 
On-the-Job  Training  (OJT)  supervisors  to  assess  trainee  proficiency  at 
specific  maintenance  or  equipment-oriented  tasks.  Procedures  for 
developing  assessment  instruments  for  non-equipment-oriented  tasks  will 
be  published  in  another  document:  Task  Evaluation  Form;  Development 
Procedures  for  Non-Equipment-Oriented  Tasks. 

The  authors  wish  to  acknowledge  the  assistance  and  cooperation  of 
the  many  individuals  who  contributed  to  the  completion  of  this 
document.  From  ASA,  Rohn  J.  Hritz  and  George  R.  Purifoy,  Jr.  for  their 
contributions  to  the  initial  derivation  of  the  Task  Evaluation  Form 
(TEF)  Development  Procedures  and  for  their  support  and  ideas  throughout 
the  project;  Ms.  Lisa  I.  Thocher  for  her  assistance  in  the  data 
collection  and  analysis;  and  the  Project  Secretary,  Mrs.  Tammy  Mowry. 
Special  thanks  are  due  to  Mr.  Gerald  S.  Walker,  the  Air  Force  Contract 
Monitor. 

This  project  would  not  have  been  possible  without  the  cooperation 
of  all  of  the  Air  Force  SMEs  and  project  coordinators.  The  authors  are 
especially  grateful  to  the  following: 


Captain  Richard  Dineen 
MSgt  Robert  Aust 

Captain  Neil  Breckenridge 

TSgt  Gilbert  Elmy 

MSgt  Goolsby 

MSgt  Thadous  J.  Magwood 
MSgt  Larry  R.  Schmidt 
CMSgt  David  H.  Smith 
Captain  Sherry  D.  Webb 
Major  Martin  Costellic 
Major  Larry  Johnston 
Major  Gary  Marti  11a 
Colonel  Charles  Daye 


AFHRL/ID,  Lowry  AFB 
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AFB 
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INTRODUCTION 


Purpose  of  the  Handbook 


This  handbook  contains  instructions  for  the  development  of  a  Task 
Evaluation  Form  (TEF).  TEFs  can  be  used  by  On-the-Job  Training  (OJT) 
supervisors  to  evaluate  the  performance  of  a  specific  task  (e.g.. 
Preflight  Inspection,  Liquid  Oxygen  [LOX]  Servicing).  This  handbook  is 
used  by  Subject  Matter  Experts  (SMEs)  to  develop  a  TEF  that  can  then  be 
used  by  all  evaluators  given  the  responsibility  to  evaluate  performance 
of  that  task.  The  TEF  is  used  for  the  evaluation  of  task  performance 
only  and  is  not  meant  for  use  in  evaluating  systems  knowledge. 

The  individual  who  develops  the  TEF  will  probably  not  be  the  same 
person  who  uses  it  to  evaluate  a  task.  The  person  using  this  handbook 
is  called  the  TEF  developer  and  will  be  responsible  for  using  the 
instructions  in  the  handbook  to  produce  a  TEF.  The  evaluator  is 
responsible  for  using  the  TEF  to  evaluate  task  performance.  Thus,  the 
developer  is  actually  providing  information  that  will  be  used  later  by 
an  evaluator. 

The  instructions  contained  in  this  handbook  will  guide  the 
developer  in  making  decisions  about  what  performer  actions  and  outcomes 
actually  reflect  successful  task  performance.  Possible  performer 
actions  and  outcomes  are  divided  into  five  evaluation  areas 
(time/speed,  sequence-following,  end  product,  safety,  and  tools, 
equipment,  and  materials  use).  The  instructions  in  this  handbook  guide 
the  developer  in  determining  which  areas  should  be  evaluated  during 
task  performance.  The  developer  considers  every  step  or  event  in  the 
task  and  decides  at  what  point  in  the  task  a  particular  evaluation  area 
should  be  evaluated.  Once  critical  task  events  or  steps  have  been 
identified,  instructions  are  provided  for  describing  successful  task 
performance.  These  descriptions  are  considered  performance  standards 
or  criteria  and  are  entered  on  the  TEF.  (Later,  the  evaluator  uses 
this  information  to  decide  whether  the  task  has  been  correctly 
performed. ) 

In  addition  to  guiding  the  developer  in  identifying  and  describing 
critical  aspects  of  task  performance,  the  TEF  development  procedures 
provide  instructions  for  creating  an  evaluation  scenario  and  for 
developing  a  chart  for  scoring  task  performance.  The  evaluation 
scenario  and  scoring  chart  are  also  entered  on  the  TEF.  The  evaluator 
uses  the  evaluation  scenario  as  instructions  for  setting  up  and 
conducting  the  evaluation.  The  scoring  chart  is  also  used  by  the 
evaluator  to  score  the  observed  task  performance. 

A  TEF  (resulting  from  the  application  of  the  procedures  contained 
in  this  handbook)  describes  the  following  for  the  evaluator: 


1.  How  to  set  up  the  evaluation  —  an  evaluation  scenario  is 
described  so  that  all  supervisors  evaluating  the  same  task  will  conduct 
the  evaluation  in  the  same  manner. 

2.  What  to  evaluate  --  the  task  events  or  steps  are  designated  on 
the  TEF  so  that  all  supervisors  evaluate  the  same  aspects  of  task 
performance. 

3.  How  to  evaluate  —  the  scoring  chart  eliminates  evaluator 
differences  in  scoring  and  assigning  pass/fail  decisions.  A  separate 
score  is  obtained  for  each  evaluation  area  based  on  the  number  and  type 
of  errors  that  occur  during  task  performance.  Explicit  criteria  for 
assigning  an  overall  pass/fail  decision  are  also  provided. 

Examples  of  completed  Task  Evaluation  Forms  are  included  in 
Appendix  A,  along  with  an  overview  of  the  forms. 


Who  Should  Use  This  Handbook 


This  handbook  was  designed  for  use  by  individuals  given  the 
responsibility  for  developing  a  TEF  for  maintenance  or  equipment- 
oriented  tasks.  It  contains  instructions  applicable  to  tasks  performed 
in  maintenance  Air  Force  Specialty  Codes  (AFSCs)  and  any  other  tasks 
that  are  primarily  equipment-oriented. 

An  alternative  handbook  (AFHRL-TP-85-56,  Task  Evaluation  Form: 
Development  Procedures  for  Non-Equipment-Oriented  Tasks)  contains 
instructions  for  non-equipment-oriented  tasks.  Individuals  from 
non-maintenance  AFSCs  should  refer  to  the  Non-Maintenance  Handbook. 
However,  when  a  TEF  is  being  developed  for  an  equipment-oriented  task, 
most  likely  the  present  handbook  will  be  applicable  regardless  of 
AFSC. 

The  developer  should  be  familiar  with  the  task  for  which  the  TEF 
is  being  developed.  In  addition,  it  is  helpful  if  the  developer  has 
had  the  responsibility  for  training  and  evaluating  in  the  OJT 
environment. 


Why  Should  Task  Evaluation  Forms  Be  Used 


During  the  Air  Force  training  experience,  an  airman  is  trained  in 
both  job  skills  and  knowledge.  The  evaluation  of  an  airman's 
proficiency  can  be  divided  into  two  separate  types  of  evaluation:  the 
evaluation  of  job  knowledge  and  the  evaluation  of  job  performance.  Job 
knowledge  is  typically  assessed  through  written  tests.  These  tests  are 


standardized  so  that  every  airman  is  tested  on  the  same  subject  matter, 
in  the  same  way.  In  addition,  a  standardized  scoring  routine  is  used 
to  score  written  tests.  Thus,  for  written  tests,  test  content, 
administration,  and  scoring  are  standardized. 

This  is  not  the  case  with  the  evaluation  of  job  skills,  which  are 
assessed  through  the  evaluation  of  task  performance.  Evaluation  of 
task  performance  is  not  as  standardized  as  is  the  evaluation  of  job 
knowledge.  Standardization  is  lacking  in  the  areas  of  evaluation 
instrument  development,  administration,  and  scoring.  In  other  words, 
differences  exist  among  supervisors  regarding  what  is  evaluated,  how 
evaluations  are  conducted,  and  how  evaluations  are  scored.  These  three 
areas  of  difference  are  discussed  below. 


What  is  Evaluated 

Different  evaluators  tend  to  focus  on  different  aspects  of 
performance  when  assessing  task  proficiency.  One  supervisor  may 
evaluate  the  performer's  ability  to  properly  use  tools,  equipment,  or 
materials.  Another  evaluator  may  focus  on  whether  the  performer 
completes  the  steps  in  the  task  in  a  prescribed  sequence.  Some 
supervisors  are  influenced  by  factors  that  are  related  to  the  performer 
(such  as  neatness  of  dress,  attitude,  and  motivation),  while  other 
supervisors  find  those  types  of  evaluation  factors  less  meaningful. 


How  is  the  Evaluation  Conducted 

In  addition  to  focusing  on  different  aspects  of  the  task  during 
the  evaluation,  evaluators  conduct  the  evaluations  in  different  ways. 
Some  supervisors  use  the  evaluation  situation  as  a  training  experience 
for  the  performer.  Supervisors  have  a  tendency  to  immediately  correct 
the  performer  when  skill  deficiencies  are  observed.  This  immediate 
feedijack  confounds  the  evaluation  process  and  results,  making  it 
difficult  to  use  the  evaluation  results  in  meaningful  ways. 

The  evaluation  situation  is  often  used  as  an  opportunity  to  test 
"systems  knowledge"  through  questions  to  the  performer.  During  task 
performance,  some  evaluators  evaluate  both  knowledge  and  skills,  while 
other  supervisors  evaluate  only  the  performer's  ability  to  successfully 
complete  the  task  (skills). 


How  are  the  Results  of  the  Evaluation  Scored 

The  third  area  of  evaluator  differences  is  in  the  scoring  of  the 
results  of  the  evaluation.  There  are  no  standard  criteria  to  define 
the  types  and  number  of  errors  a  performer  is  allowed  to  make  and  still 
be  considered  qualified  to  perform  the  task.  Criteria  for  scoring  an 
evaluation  are  also  lacking.  An  evaluator  has  no  means  of  arriving  at 


a  meaningful  final  score  for  task  performance.  This  makes  it  difficult 
to  compare  several  different  performers  or  to  compare  the  same 
individual's  performance  at  different  points  in  time. 


The  ability  to  accurately  and  fairly  assess  an  individual's 
ability  to  perform  a  task  is  important  within  the  OJT  system. 

Currently,  the  OJT  supervisor  is  responsible  for  the  evaluation  of 
trainee's  task  performance. 

The  use  of  TEFs  will  help  to  ensure  that  all  evaluators  when 
evaluating  a  task,  will; 

1.  Conduct  the  evaluation  the  same  way. 

2.  Evaluate  the  same  performer  actions  and  outcomes. 

3.  Use  the  same  criteria  to  decide  whether  the  task  was  correctly 
performed. 

4.  Score  task  performance  the  same  way. 


Overview  of  the  Task  Evaluation  Form  Development  Procedures 


Throughout  the  TEF  development  process,  instructions  are  provided 
for  entering  information  onto  worksheets.  These  worksheets  are 
included  as  a  tool  for  the  developer.  At  the  end  of  TEF  development, 
the  information  from  the  worksheets  will  be  transferred  onto  the  TEF. 
Only  the  resulting  TEF  is  used  by  the  evaluator. 

The  development  process  is  organized  around  the  completion  of  nine 
worksheets.  The  handbook  provides  instructions  for  completing  each 
worksheet  as  follows: 


1. 

Worksheet 

01 

Defining  the  Task. 

2. 

Worksheet 

02 

Listing  the  Task  Steps. 

3. 

Worksheet 

03 

Evaluation  of  Time  or  Speed  of  Task 

Performance. 

4. 

Worksheet 

04 

Evaluation  of  Sequence-Following. 

5. 

Worksheet 

05 

Evaluation  of  End  Product. 

6. 

Worksheet 

06 

Evaluation  of  Safety  Procedures  and 

Regulations. 

7. 

Worksheet 

07 

Evaluation  of  Tools,  Equipment,  and  Materials 
Use. 

8. 

Worksheet 

08 

Evaluation  Scenario. 

9. 

Worksheet 

09 

Scoring  Criteria. 

Worksheets  01 

and 

02  describe  the  task  that  is  to  be  performed  for 

the  evaluation.  Worksheets  03  through  07  describe  the  aspects  of  task 
performance  that  should  be  evaluated.  Worksheet  08  contains  the 


description  of  the  evaluation  scenario,  and  Worksheet  09  describes  how 
task  performance  should  be  scored. 

For  each  worksheet,  the  following  general  information  is 
provided: 

1.  An  overall  explanation  of  the  information  that  will  be 
entered. 

2.  The  purpose  of  the  information  from  the  evaluator's  point  of 

view. 

3.  Examples  of  completed  worksheets  for  two  tasks. 

In  addition,  specific  instructions  are  provided  for  completing  each 
individual  item  on  the  worksheets.  These  instructions  include 
directions  for  selecting  the  appropriate  information  and  guidelines  for 
entering  that  information. 


Getting  Started 


To  get  started,  the  developer  should  have  a  specific  task  in  mind. 
Once  a  task  has  been  selected,  the  developer  begins  with  Worksheet  01. 

Blank  worksheets  are  contained  in  Appendix  B.  Before  TEF 
development,  it  will  probably  be  necessary  to  make  extra  copies  of  the 
worksheets. 
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WORKSHEET  01:  LISTING  THE  TASK  STEPS 


Overview 


Explanation 

In  order  to  develop  a  TEF,  a  list  of  the  steps  in  the  task  is 
necessary.  The  steps  in  the  task  are  simply  what  someone  does  to 
perform  the  task.  The  development  procedures  provide  instructions  for 
the  development  of  a  TEF;  they  do  not  provide  instructions  for 
generating  a  list  of  the  task  steps.  You  are  expected  to  obtain  the 
list  from  another  source  and  enter  the  steps  on  Worksheet  01. 


Purpose 

You  (the  developer)  will  refer  to  the  task  steps  throughout  the 
TEF  development  process.  Later,  you  will  actually  be  describing 
acceptable  performance  for  many  of  the  steps. 

When  the  list  is  transferred  to  the  TEF,  the  evaluator  will  use 
the  task  steps  as  a  reference  point  during  the  actual  evaluation. 


Completing  Worksheet  01 

To  complete  Worksheet  01,  you  will: 

1.  List  the  steps  on  Worksheet  01. 

2.  Number  the  steps. 

Examples  of  Worksheet  01 

Examples  of  completed  Worksheet  01  are  shown  in  Figures  1  and 


2. 


Worksheet  01:  Task  Steps 

Task:  LOX  Servicing  B-52H  Developer:  Sample 

Column  A:  # 

Column  B:  Step  Description 

1 

2 

Attach  two  cables  from  aircraft  to  static  grounds. 

3 

Position  fire  extinguisher. 

4 

Disconnect  electrical  power  to  aircraft. 

5 

Position  battery  power  switch  to  off. 

6 

Ensure  sufficient  LOX  is  in  service  cart. 

7 

Position  LOX  service  cart  at  far  limit  of  hose. 

8 

Set  brakes  and  chock  wheels  of  LOX  cart. 

9 

Attach  LOX  cart  grounding  cable  to  static  ground. 

10 

Ensure  area  around  A/C  vent  is  clean  and  clear. 

11 

Position  clean  metal  drip  pans  and  purge  cart  vent. 

12 

Open  filler  valve  access  door. 

13 

Don  protective  clothing. 

14 

Clear  personnel  from  vent  area. 

15 

Position  vent  valve  handle  to  Fill  position. 

16 

Ensure  filler  valve  compartment  is  clean  and  clear. 

17 

Remove  filler  valve  cap  and  clean  filler  valve. 

18 

Ensure  cart  hose  nozzle  and  purge  fittings  are  dry 
and  clean. 

19 

Ensure  capacity  gage  equalizer  is  open. 

20 

Build  up  service  cart  pressure  to  30  +  5  psi. 

Figure  1.  Example  of  Completed  Worksheet  01: 

Liquid  Oxygen  (LOX)  Servicing  B-52H  Aircraft- 
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Worksheet  01:  Task  Steps 


LOX  Servicing  B-52H 


Developer:  Sample 


Column 


Column  B:  Step  Description 


Check  operation  of  cart  relief  valve. 


Open  service  cart  supply  valve. 


Purge  supply  hose. 


ressurize  hose. 


Connect  supply  hose  to  aircraft  filter  valve. 


Open  service  cart  supply  valve  and  maintain  30+5 
si . 


Close  service  cart  supply  valve. 


Disconnect  supply  hose  from  aircraft. 


Depressurize  supply  hose. 


Replace  aircraft  filler  valve  cap. 


Position  aircraft  vent  valve  handle  to  build  up 
position. 


Close  filler  valve  access  door  and  secure. 


Depressurize  service  cart. 


Remove  LOX  cart  from  area. 


Connect  main  external  power  to  aircraft. 


Check  LOX  quantit 


Disconnect  main  external  power  from  aircraft. 


38 


Disconnect  extra  grounding  cable. 


Task:  Prefliqht  KC-135A 


Worksheet  01:  Task  Steps 


Developer:  Sample 


Column  B;  Step  Description 


Test  fuel  quantity  gages. _ 


Check  pitot  tubes,  check  angle  of  attack,  check 
static  boom  tubes. 


Check  rendezvous  beacon  lights  for  o 


Check  navigation  lights  for  operation. 


Check  LOX 


e  readin 


Depressurize  and  check  hydraulic  quantit 


Close  circuit  breakers:  engine  fuel  flow  and  EPR 
indicators. 


Check  boom  nozzle  light  bulb  for  o 


Check  portable  fire  extinguisher. 


Check  forward  chinning  bar  safety  pin  installed. 


Check  portable  oxygen  cylinders  for  operation, 
pressure,  and  stowed. 


Check  nose  gear  inspection  window. 


Check  taxi  lights 


Check  landing  lights. 


Check  pilot  and  co-pilot  windows. 


Cargo  compartment 

Check  portable  oxygen  cylinders  for  operation, 
ressure,  and  stowed. 


Check  portable  fire  extinguishers  properly  secured. 


Check  gaseous  oxygen  system  serviced. 


Check  main  gear  and  flap  emergency  crank  stowed. 


Column  A:  # 


Column  B:  Step  Description 


Check  main  gear  inspection  window. 


Check  escape  spoiler  air  supply  bottle  properly 
serviced. 


Check  AIMS  static  system  drained  of  moisture. 


Check  nose  gear  strut  for  proper  extension. 


Check  nose  gear  tires. 


Check  static  ports  for  obstruction. 


Visually  inspect  lower  fuselage  for  fuel  or  hydraul 
leaks. 


Columns  A  and  B  Step  Number  and  Description 


Sources  of  Task  Steps 

Some  recommended  sources  of  task  steps  are: 

1.  Task  analyses  (e.g..  Automated  Task  Analysis  Authoring  Aid) 

2.  Technical  orders 

3.  Work  cards 

4.  Checklists 

5.  OJT  manuals 


Criteria  for  Task  Steps 


1.  Task  steps  should  be  written  in  the  same  order  they  are 
usual ly  performed. 

2.  Task  steps  should  be  written  in  easy-to-understand  terms. 

3.  Task  steps  should  not  be  lengthy. 

4.  Task  steps  should  simply  describe  what  someone  does  to  perform 
the  task. 


5.  Task  steps  should  not  describe  how  someone  performs  the  task 
(they  should  not  describe  standards  or  criteria  for  acceptable 
performance;  these  will  be  described  later). 

6.  Task  steps  should  be  numbered  so  they  can  be  referred  to  by 
number  as  the  worksheets  are  completed. 

7.  Task  steps  should  be  in  accordance  with  approved  technical 

data. 


Guidelines  for  Entering  Task  Steps 

The  task  steps  should  be  entered  on  Worksheet  01.  Be  sure  to 
number  the  steps. 

When  one  of  the  task  documentation  sources  (e.g.,  technical 
orders,  checklists,  etc.)  contains  an  acceptable  list  of  task  steps, 
you  do  not  have  to  rewrite  all  of  the  steps.  Instead,  you  should: 

1.  Review  the  documentation  source. 

2.  Check  the  list,  to  make  sure  it  is  complete. 

3.  Check  the  list,  to  make  sure  the  steps  are  easy  to 
understand. 


CAN  THE  TASK  BE  EVALUATED  BY  A  TASK  EVALUATION  FORM? 


Overview 


Explanation 


The  purpose  of  this  discussion  is  to  determine  if  the  task  of 
interest  can,  in  fact,  be  evaluated  by  a  TEF.  If  not,  then  it  makes  no 
sense  to  continue  with  the  remainder  of  this  handbook.  Three 
evaluation  methods  will  be  discussed.  If  none  of  these  methods  can  be 
used  to  evaluate  the  task  of  interest,  then  you  should  abandon  the 
development  of  the  TEF  and  select  another  task.  Again,  the  purpose  of 
this  step  is  not  to  select  the  "best"  method  of  evaluation,  but  simply 
to  determine  if  the  task  of  interest  can  be  evaluated  using  any  or  all 
of  the  three  methods  discussed  below.  There  is  no  worksheet  associated 
with  this  discussion. 


Discussion  of  Evaluation  Methods 


There  are  basically  three  ways  to  conduct  an  evaluation  with  a 

TEF: 


1.  Method  1.  Use  Actual  Equipment  in  Job  Environment.  The 
operational  equipment  (e.g.,  the  aircraft)  can  be  used  as  it  actually 
exists  in  the  job  environment.  For  example,  to  evaluate  a  preflight 
inspection  task,  an  aircraft  on  the  flightline  that  requires  a 
preflight  inspection  can  be  used.  The  performer  and  evaluator  would  go 
to  the  aircraft  and  the  performer  would  perform  the  preflight 
inspection  while  the  evaluator  observed. 

This  method  is  appropriate  when  the  task  occurs  frequently  in 
the  job  environment,  and  the  evaluation  of  task  performance  does  not 
interfere  with  successful  completion  of  the  task, 

2.  Method  2.  Use  "Rigged"  Actual  Equipment.  The  actual 
operational  equipment  can  be  "rigged"  in  some  way  to  present  the 
desired  situation.  For  example,  suppose  the  task  occurs  infrequently 
(too  infrequently  for  the  evaluator  to  wait  for  the  task  to  be 
performed  on  the  flightline).  In  this  situation,  it  might  be 
convenient  for  the  evaluator  to  "rig"  the  aircraft  (or  operational 
equipment)  so  that  the  task  of  interest  can  be  performed  and  evaluated. 
Some  emergency  tasks  can  be  evaluated  this  way.  This  method  is 
different  than  the  first  evaluation  method  even  though  both  methods  use 
operational  equipment. 


V-'J 
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3.  Use  Trainer  or  Simulator.  If  the  task  of 
does  not  occur  l^requently  in  the  job  situation  and  the  aircraft 
(operational  equipment)  cannot  be  "rigged"  because  "rigging"  the 
aircraft  will  create  a  hazardous  situation,  then  the  only  alternative 
is  to  evaluate  the  task  on  a  simulator  or  trainer. 

This  method  is  appropriate  when  a  trainer  or  simulator  is 
available  and  the  task  can  be  presented  on  the  trainer  or  simulator. 

The  purpose  of  this  step  in  the  process  is  to  determine  if  any  of 
these  evaluation  methods  can  be  used.  It  is  not  to  select  one  method, 
but  only  to  assure  yourself  that  the  task  of  interest  can  be  evaluated 
by  one  of  the  three  methods. 


Determining  Possibility  of  Evaluation  With  a  TEF 


To  determine  if  the  task  under  consideration  can  be  evaluated 
using  a  TEF,  ask  yourself  the  following  questions: 


Questions  on  Method  1 

1.  Is  the  task  under  consideration  performed  frequently  in  the 
job  environment? 

NO  -  Go  to  Method  2. 

YES  -  Go  to  the  next  question. 

2.  Will  the  evaluation  of  task  performance  interfere  with 
successful  completion  of  the  task? 

NO  -  Method  1  will  probably  be  appropriate.  You  will  select 
the  best  method  later. 

YES  -  Go  to  Method  2. 


Questions  on  Method  2 

1.  Is  it  possible  to  "rig"  the  operational  equipment  to  present 
the  desired  task? 

NO  -  Go  to  Method  3. 

YES  -  Go  to  the  next  question. 


2.  Is  it  possible  to  "rig"  the  aircraft  without  creating  a 
hazardous  situation? 

NO  -  Go  to  Method  3. 

YES  -  Method  2  wi 1 1  probably  be  appropriate.  You  will  select 
the  best  method  later. 


Questions  on  Method  3 

1.  Is  a  trainer  or  simulator  available? 

NO  -  It  is  probably  not  possible  to  evaluate  the  task  under 
consideration  by  an  over-the-shou Ider  evaluation.  A  TEF  cannot  be 
developed  for  this  task. 

YES  -  Go  to  the  next  question. 

2.  Can  the  task  be  presented  on  the  t>a'npr  s’'^.jlator? 

NO  -  It  is  probably  not  poss'D’-  t  “ .  \  jnder 

consideration  by  an  over-the-shoulder  eva  .a*'  '  -  '•  anno*  be 

developed  for  this  task, 

YES  -  Method  3  wi  11  probab'  v  Ot-  ac;-  .  '  select 

the  best  method  later. 

Based  on  your  answers  to  the  quei?  ■  "a--  !  v  ■,  m  about  the 

possi  bi  1  i  ty  of  evaluating  the  task  t  n  a  ' :  •  .  a '  .  ■  ‘  t  he  task 

is  performed  frequently  in  the  job  env '  f  n  ..  oe  nc 

problem.  However,  if  the  task  occurs  •  nf ,  •  "t-  -^Ui  pment 
cannot  be  "rigged,"  and  a  trainer  ■.  s  not  a.a'  ab  'hen  t  is  probably 
not  worthwhile  to  continue  with  the  rest  t  n*-  'f-  teve’opment 
procedures. 

If  your  decision  is  that  the  tasx  canf)()r  be  ev-a  iuted  by  any  of 
the  three  methods,  then  record  a  note  an  worksheet  01.  You  should 
simply  state,  "This  task  cannot  be  evaluated  by  any  of  the  three 
over-the-shoulder  eval uat ion  methoas. "  You  should  select  another 
task. 

If  your  decision  is  that  the  task  can  he  evaluated,  go  to 
Worksheet  02,  and  a  specific  method  of  evaluation  wi 1 1  be  selected 
later.  You  cannot  select  a  specific  method  now,  since  you  need  more 
information  about  what  should  be  evaluated  during  task  performance. 


WORKSHEET  02:  DEFINING  THE  TASK 


Overview 


Explanation 

The  first  step  in  TEF  development  is  to  define  the  particular  task 
for  which  the  form  is  being  developed.  Individual  TEFs  will  probably 
be  developed  for  many  tasks.  Therefore,  a  description  of  the  task 
that  should  be  evaluated  with  the  TEF  you  are  about  to  develop  is 
necessary.  The  task  title  may  not  provide  enough  information  to 
completely  identify  the  task  to  be  evaluated.  You  will  identify 
exactly  what  steps  or  events  should  be  considered  part  of  the  task  for 
evaluation  purposes.  In  addition,  when  there  are  multiple  performers, 
you  will  specify  which  performer  should  be  evaluated  with  this  TEF. 


Purpose 

In  order  to  conduct  fair  evaluations,  it  is  important  that  all 
evaluators  have  the  same  idea  about  what  is  to  be  included  in  the 
evaluation.  Worksheet  02  outlines  the  task  to  be  evaluated.  The 
information  on  this  worksheet  tells  the  evaluator  exactly  what  steps  or 
parts  of  the  overall  task  should  be  included  in  the  evaluation;  i.e., 
what  steps  in  the  task  the  performer  is  responsible  for. 


Completing  Worksheet  02 

In  order  to  complete  Worksheet  02,  you  will  enter  the  following 
i nformation: 


1. 

Line  A 

AFSC/Duty  Position 

or  Work  Center. 

2. 

Line  B 

Task  Title. 

3. 

Line  C 

Task  Beginning. 

4. 

Line  D 

Task  End. 

5. 

Block  E 

Steps  or  Events  not  Included  in  the  Evaluation. 

6. 

Block  F 

Source  Title  and 

Identification. 

Examples  of  Worksheet  02 

Examples  of  completed  worksheets  are  shown  in  Figures  3  and  4. 


Worksheet 


Worksheet  02;  Task  Definition 


On  Line  A  you  will  enter  the  AFSC  and  duty  position  or  work  center 
for  which  the  TEF  is  being  developed. 


Purpose 

The  AFSC  provides  general  information  regarding  who  should  be 
evaluated.  The  duty  position  or  work  center  further  defines  the 
performer's  role  for  the  evaluator.  This  is  especially  important  when 
the  task  is  a  team  task  or  when  several  people  perform  different  parts 
of  the  task.  This  information  tells  the  evaluator  which  member  of  the 
team  should  be  evaluated  with  this  particular  TEF. 


Guidelines  for  Entering  AFSC  and  Duty  Position 

Simply  enter  the  AFSC  and  duty  position  or  work  center  in  Block  A. 
You  do  not  have  to  include  a  work  center  or  duty  position  for  every 
task.  You  should  include  either  the  duty  position  or  work  center  for 
tasks  that  usually  require  more  than  one  person  for  completion. 

Some  examples  of  duty  positions  are: 

1.  Task  supervisor 

2.  Crew  chief 

3.  Operator 

4.  Servicing  unit  monitor 

5.  Aircraft  systems  gage  monitor 

6.  Team  chief 


Line  B  Task  Title 


Exp  1 anation 

On  Line  B  you  will  enter  the  title  of  the  task,  including  the 
weapon  system  when  applicable. 


Purpose 

When  the  evaluator  is  ready  to  evaluate  the  performance  of  a 
particular  task,  the  appropriate  TEF  can  be  found  by  checking  the 
title. 


Enter  the  task  title  in  Block  B.  Be  sure  to  include  the  weapon 
system  or  name  of  the  operational  equipment  (when  applicable). 

Some  examples  of  task  titles  are  shown  below: 

1.  Drag  Chute  Installation  -  B-52H 

2.  Preflight  -  KC-135A 

3.  LOX  Servicing  -  B-52H 

4.  Starter  Cartridge  Remove  -  KC-135A 


Line  C  Task  Beginning 


Explanation 


The  task  beginning  is  the  exact  point  at  which  the  evaluation  of 
task  performance  begins.  For  instance,  if  the  task  to  be  evaluated 
requires  a  lengthy  set-up  procedure  and  the  set-up  procedure  is 
considered  a  whole  different  task,  then  the  evaluation  would  begin 
after  the  set-up  procedure  had  been  completed. 


Purpose 


The  information  on  Line  C  will  tell  the  evaluator  when  to  begin 
the  evaluation. 


Guidelines  for  Entering  Task  Beginning 

Enter  the  task  beginning  in  Block  C  of  Worksheet  02. 


You  can  enter  a  task  step  number  or  a  performer  action  or  event 
which  the  evaluator  can  observe. 


Some  examples  of  acceptable  entries  are  shown  below: 


1.  As  the  performer  positions  B-4  stand. 

2.  Step  3. 

3.  Step  1. 

4.  After  the  LOX  cart  is  set-up. 

5.  As  the  performer  positions  the  fire  extinguisher. 

6.  As  the  performer  opens  the  access  doors. 

7.  As  the  performer  receives  clearance. 


Line  0  Task  End 


Explanation 

The  task  end  is  the  exact  point  at  which  the  evaluation  of  task 
performance  ends. 


Purpose 

The  information  on  Line  D  tells  the  evaluator  when  to  end  the 
evaluation. 


Guidelines  for  Entering  Task  End 


Enter  the  task  end  in  Block  D. 

You  can  enter  a  step  number  or  a  performer  action  or  event  that 
the  evaluator  can  observe. 

Some  examples  of  acceptable  entries  are  shown  below: 

1.  After  B-4  stand  is  removed. 

2.  When  Step  30  is  completed. 

3.  When  Step  25f  is  completed. 

4.  After  external  electric  power  is  disconnected. 

5.  After  the  master  refuel  switch  is  positioned  to  off. 

6.  After  access  doors  are  closed. 


Block  E  Steps  or  Events  Not  Included  in  the  Evaluation 


Exp  1  anation 

There  are  times  when  the  performance  of  the  task  of  interest  may 
overlap  into  the  performance  of  other  tasks,  or  someone  other  than  the 
performer  may  be  responsible  for  certain  parts  of  the  task.  When  this 
occurs,  the  performer  should  not  be  evaluated  on  these  parts  of  the 
task.  For  example,  during  a  preflight  inspection,  the  performer  will 
service  the  LOX  if  it  is  needed.  However,  if  you  were  developing  a 
TEF  for  the  preflight,  you  might  want  to  exclude  servicing  the  LOX, 
since  this  is  a  separate  task.  In  that  case,  the  preflight  performer 
should  be  evaluated  only  on  checkin.-j  the  LOX  and  recording  any 
abnormalities.  You  would  instruct  the  evaluator  not  to  evaluate  LOX 
servicing  while  evaluating  a  preflight  inspection. 


Purpose 


Block  E  describes  for  the  evaluator  the  parts  of  the  task  that  the 
performer  is  not  responsible  for.  This  tells  the  evaluator  not  to 
evaluate  the  performer  on  these  parts. 


Finding  Steps  or  Events  Which  Should  Not  Be  Evaluated 


Think  about  the  task,  and  ask  yourself  the  following  questions: 

1.  Are  there  any  parts  of  the  task  for  which  the  performer  should 
not  be  responsible? 

2.  Are  there  any  parts  of  the  task  that  should  be  performed  by 
someone  else  (e.g. ,  by  the  evaluator  or  another  member  of  the  team)? 

3.  Are  there  any  parts  of  the  task  that  are  really  another  task? 

4.  Are  there  any  parts  of  the  task  that  should  be  evaluated  by  a 
different  TEF? 


Guidelines  for  Entering  Steps  or  Events  Which  Should  Not  Be  Evaluated 


Enter  the  steps  or  events  that  should  not  be  evaluated  in  Block  E 
You  should  make  your  entries  by  step  number  or  by  describing  the 
events. 

Some  examples  of  steps  or  events  that  should  not  be  evaluated 
(from  several  tasks)  are  shown  below: 

1.  Placing  the  generator  on  line. 

2.  Grounding  the  aircraft. 

3.  Any  servicing. 

4.  Wing  walker  duties. 

5.  Brake  rider  duties. 

6.  Fire  guard  duties. 

7.  Ground  observer  duties. 

8.  Steps  1  through  4. 

9.  Steps  7,  10,  and  11. 


Block  F  Source  Title  and  Identification 


Explanation 


In  Block  F,  you  should  list  any  task  information 
;e  to  develop  the  TEF. 


sources  you  pi, 


Now  that  you  have  defined  the  task,  you  should  select  any  relevant 
information  about  the  task.  You  will  be  using  this  task  information  as 
a  reference  or  documentation  source  during  the  development  of  the  TEF. 
The  task  information  should  be  up  to  date  and  include  the  following: 

1.  Task  steps  and  sequences  of  steps. 

2.  Tools,  equipment,  and  materials  used. 

3.  Applicable  procedures  and  regulations. 

Some  relevant  sources  of  task  information  to  consider  are: 

1.  Technical  orders. 

2.  Technical  manuals. 

3.  OJT  manuals. 

4.  Occupational  survey  data. 

5.  Logistical  support  analysis  data. 

Review  all  of  the  available  information  sources  and  decide  which 
sources  provide  the  most  complete  information.  Select  the  task 
information  sources  that  you  will  use  to  develop  the  TEF. 


Guidelines  for  Entering  Task  Information  Sources 

Enter  the  titles  and  the  date  of  your  sources  in  Block  F.  (If  you 
are  entering  technical  orders,  write  the  date  of  the  latest  change.) 


YOU  ARE  NOW  READY  TO  GO  ON  TO  WORKSHEET  03. 


WORKSHEET  03:  EVALUATION  OF  TIME  OR  SPEED  OF  TASK  PERFORMANCE 


Overview 


Explanation 

The  evaluation  of  time  or  speed  of  task  performance  is  the 
evaluation  of  the  amount  of  time  the  performer  is  allowed  to  complete  a 
task  or  part  of  a  task.  One  of  the  indications  of  successful  task 
performance  may  be  whether  the  performer  completes  the  task  or  part  of 
a  task  within  a  specified  time  period. 


Purpose 

The  information  you  enter  on  Worksheet  03  will  describe  for  the 
evaluator: 

1.  Exactly  when  in  the  task  the  time  or  speed  of  task  performance 
should  be  evaluated. 

2.  The  amount  of  time  which  is  acceptable. 

The  evaluator  uses  this  information  to  decide  whether  the  task  or 
parts  of  the  task  were  performed  in  the  correct  amount  of  time. 


Completing  Worksheet  03 


In  order  to  complete  Worksheet  03,  you  will  enter  the  following 
i nformation: 


1. 

Column 

A 

Critical 

Segments. 

2. 

Column 

B 

Starting 

Point. 

3. 

Column 

C 

Stopping 

Point. 

4. 

Column 

D 

Standard. 

Examples  of  Worksheet  03 


See  Figures  5  and  6  for  examples  of  completed  Worksheet  03. 


Worksheet  03;  Evaluation  of  Time  or  Speed  of  Task  Performance 


Worksheet  03;  Evaluation  of  Time  or  Speed  of  Task  Performance 


Figure  6 


Example  of  Completed  Worksheet  03; 
Preflight  KC-135A. 
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In  Column  A,  you  should  enter  the  critical  segments.  The  critical 
segments  are  those  steps  or  series  of  steps  in  the  task  where  the  time 
or  speed  of  task  performance  should  be  evaluated. 


Purpose 

The  list  of  critical  segments  tells  the  evaluator  when  to  evaluate 
the  time  or  speed  of  performance. 


Finding  Critical  Segments 

Time  or  speed  of  performance  can  be  evaluated  for; 

1.  A  step  in  the  task. 

2.  A  series  of  steps  in  the  task. 

3.  The  whole  task. 


Time  or  speed  of  performance  may  not  be  applicable  to  the  task  under 
consideration. 


You  should  decide,  by  using  the  criticality  questions  included  in 
this  section,  where  in  the  task  time  or  speed  of  performance  should  be 
evaluated.  You  should  go  through  each  step  in  the  task  and  ask 
yourself,  "Will  any  of  the  following  occur  if  this  step  or  series  of 
steps  is  not  performed  within  some  time  period?" 

1.  Injury  to  personnel. 


2.  Damage  or  loss  of  equipment  or  resources. 

3.  Performer  required  to  redo  the  task  or  part  of  the  task. 

4.  Co-worker  (working  with  the  performer  on  the  same  task) 
delayed  in  initiating  or  completing  a  task  segment. 


REMEMBER  THAT  TIME  OR  SPEED  OF  PERFORMANCE  SHOULD  ONLY  BE 
EVALUATED  WHEN  IT  IS  CRITICAL  ACCORDING  TO  THE  CRITICALITY  QUESTIONS. 


The  critical  segments  should  be  entered  in  Column  A  of  Worksheet 
03.  Use  the  appropriate  guideline  from  the  list  below. 

1.  If  time  or  speed  of  performance  is  not  applicable  to  the  task, 
enter  N/A  in  Column  A.  Go  on  to  Worksheet  04. 

2.  If  time  or  speed  of  performance  should  be  evaluated  for  a 
step,  enter  the  step  number  (e.g.,  3,  5,  7,  9). 

3.  If  time  or  speed  of  performance  should  be  evaluated  for  a 
series  of  steps,  enter  the  first  step  number  through  the  last  step 
number  (e.g.,  2  through  6,  8  through  11). 

4.  If  time  or  speed  of  performance  should  be  evaluated  for  the 
whole  task,  enter  the  first  step  number  through  the  last  step  number 
(e.g.,  1  through  36,  1  through  29). 


Column  B  Starting  Point 


Explanation 

The  starting  point  is  the  exact  point  at  which  the  evaluator 
should  begin  timing  performance.  A  starting  point  should  be  described 
for  every  critical  segment  listed  in  Column  A. 


Purpose 


The  starting  point  tells  the  evaluator  exactly  when  to  begin 
timing  the  performer.  In  order  to  conduct  fair  and  accurate 
evaluations,  it  is  important  that  all  evaluators  who  use  the  TEF  begin 
timing  performance  at  the  same  point  in  the  task. 


Guidelines  for  Entering  Starting  Point 

Your  description  of  the  starting  point  should  be  entered  in  Column 
B  of  Worksheet  03. 

1.  You  should  describe  the  event  or  performer  action  that  would 
signal  the  evaluator  to  start  timing  task  performance.  The  event  or 
performer  action  which  you  describe  must  be  clearly  observable  to  the 
evaluator. 


2.  Your  description  should  be  brief,  but  provide  enough 
information  so  that  the  evaluator  knows  when  to  start  measuring  time. 


3.  You  may  want  to  start  your  description  with  one  of  the 
following  phrases: 

a.  Begin  when. 

b.  Begin  as. 

c.  Begin  after. 

REMEMBER,  YOUR  DESCRIPTION  SHOULD  TELL  THE  EVALUATOR  EXACTLY  WHEN 
TO  BEGIN  TIMING  PERFORMANCE. 

Some  examples  of  acceptable  entries  from  various  tasks  are: 

1.  Begin  when  performer  opens  supply  valve. 

2.  Begin  when  performer  grasps  vent  valve  handle. 

3.  Begin  as  LOX  stream  begins  to  flow. 


Column  C  Stopping  Point 


Explanation 


The  stopping  point  is  the  exact  point  at  which  the  evaluator 
should  stop  timing  performance.  A  stopping  point  should  be  described 
for  each  critical  segment  listed  in  Column  A. 


Purpose 


The  stopping  point  tells  the  evaluator  exactly  when  to  stop 
timing  the  performer.  In  order  to  conduct  fair  and  accurate 
evaluations,  it  is  important  that  all  evaluators  using  the  TEF  stop 
timing  performance  at  the  same  point  in  the  task. 


Guidelines  for  Entering  Stopping  Point 

The  stopping  point  should  be  entered  in  Column  C  of  Worksheet  03. 

1.  You  should  describe  the  event  or  performer  action  that  would 
signal  the  evaluator  to  stop  timing  performance.  The  event  or 
performer  action  you  describe  should  be  clearly  observable  to  the 
evaluator. 


2.  Your  description  should  be  brief,  but  should  provide  enough 
information  so  that  the  evaluator  knows  when  to  stop  measuring  time. 


3.  You  may  want  to  start  your  description  with  one  of  the 
following  phrases: 

a.  Stop  when. 

b.  Stop  as. 

c.  Stop  after. 

REMEMBER,  YOUR  DESCRIPTION  SHOULD  TELL  THE  EVALUATOR  EXACTLY  WHEN 
TO  STOP  TIMING  PERFORMANCE. 

Some  examples  of  stopping  points  from  various  tasks  are: 

1.  Stop  when  supply  valve  is  closed. 

2.  Stop  when  vent  valve  handle  is  in  build-up  position. 

3.  Stop  after  LOX  stream  is  discontinued. 


Column  0  Standard 


Explanation 

The  standard  is  the  amount  of  time  that  is  acceptable  to  perform 
each  critical  segment.  A  standard  should  be  entered  for  each  critical 
segment  listed  in  Column  A. 


Purpose 

The  evaluator  will  use  the  standard  to  decide  whether  the  critical 
segment  was  performed  in  an  acceptable  amount  of  time. 


Guidelines  for  Entering  Standard 

Enter  the  standard  in  Column  D  of  the  Worksheet. 

Standards  can  be  written  in  several  different  formats.  Select  the 
best  format  and  fill  in  the  appropriate  numbers. 

1.  As  a  single  value  (e.g.,  10  minutes,  15  minutes). 

2.  As  a  range  of  time.  When  you  specify  a  range  of  time,  this 
means  the  time  for  performance  must  fall  within  that  range  (e.g.,  10  to 
12  minutes,  15  to  20  minutes). 


3.  As  a  minimum  amount  of  time  (e.g.,  at  least  10  minutes,  at 
least  20  minutes) . 

4.  As  a  maximum  amount  of  time  (e.g.,  no  more  than  10  minutes 
more  than  20  minutes). 


YOU  ARE  NOW  READY  TO  60  TO  WORKSHEET  04. 


WORKSHEET  04:  EVALUATION  OF  SEQUENCE-FOLLOWING 


Overview 


Explanation 


The  evaluation  of  sequence-following  is  the  evaluation  of  the 
order  of  task  performance.  Specifically,  the  evaluation  of 
sequence-following  involves  the  order  in  which  the  steps  in  the  task 
are  performed.  Sometimes,  it  may  be  important  to  perform  a  step  or  a 
sequence  of  steps  only  after  another  step  is  performed.  Successful 
task  performance  could  be  prevented  if  the  performer  does  not  follow  a 
prescribed  order. 


Purpose 


The  information  from  Worksheet  04  describes  for  the  evaluator  the 
order  in  which  the  steps  in  the  task  should  be  performed.  The 
evaluator  uses  this  information  to  decide  whether  the  steps  were 
performed  in  the  correct  order. 


Completing  Worksheet  04 

To  complete  Worksheet  04,  you  will  enter: 

Block  A:  Series  of  steps  in  the  task  that  must  be  performed  in 
order. 

Block  B:  Single  steps  in  the  task  that  must  be  performed 
before  other  steps. 

Examples  of  Worksheet  04 

See  Figures  7  and  8  for  completed  examples  of  Worksheet  04. 


Worksheet  04:  Evaluation  of  Sequence-Following 
Task:  LOX  Servicing  B-52H _ Developer:  Sample 

Block  A  Series  of  Steps 

Step _ ^  through  38 

Step  _  through  _ 

Step  _  through  _ 

Step  _  through  _ 

Step  _  through  _ 


Block  B  Single  Steps  N/A 

Step _ before  Step(s)  _ 

Step _ before  Step(s)  _ 

Step _ before  Step(s)  _ 

Step _ before  Step(s)  _ 

Step _ before  Step(s) 


Worksheet  04:  Evaluation  of  Sequence-Following 


Task:  Preflight  KC-135A _ Developer:  Sample 

Block  A  Series  of  Steps  N/A 

Step  _  through  _ 

Step  _  through  _ 

Step  _  through  _ 

Step  _  through  _ 

Step  _  through  _ 


Block  B  Single  Steps  N/A 

Step ]_  before  Step(s)  23 

Step  _ before  Step(s)  _ 

Step _ before  Step(s)  _ 

Step _ before  Step(s)  _ 

Step _ before  Step(s)  _ 


In  Block  A  you  should  enter  any  series  of  steps  that  must  be 
performed  entirely  in  order.  (A  series  of  steps  is  three  or  more  steps 
that  must  be  performed  in  order.) 


Purpose 

The  evaluator  will  use  this  information  to  decide  whether  the 
steps  of  the  task  were  performed  in  order. 


Finding  Critical  Series 

Sequence-following  is  rarely  critical  to  evaluate  for  every  step 
in  the  task.  Many  task  reference  sources  indicate  that  the  entire  task 
must  be  performed  in  a  prescribed  sequence.  However,  quite  often 
sequence-following  should  only  be  evaluated  for  certain  series  of  steps 
in  the  task. 

You  should  use  the  criticality  questions  to  decide  when 
sequence-following  should  be  evaluated.  Go  through  the  task  and  ask 
yourself,  "Will  any  of  the  following  occur  if  certain  series  of  steps 
in  the  task  are  not  performed  in  a  specified  order?" 

1.  Injury  to  personnel. 

2.  Damage  or  loss  to  equipment  or  resources. 

3.  Performer  required  to  redo  the  task  or  part  of  the  task. 

4.  Co-worker  (working  with  the  performer  on  the  same  task) 
delayed  in  initiating  or  completing  a  task  segment. 

REMEMBER,  SEQUENCE-FOLLOWING  SHOULD  ONLY  BE  EVALUATED  WHEN  IT  IS 
CRITICAL  ACCORDING  TO  THE  CRITICALITY  QUESTIONS. 


Guidelines  for  Entering  Series  of  Steps 

Enter  the  series  of  steps  that  must  be  performed  in  order  in  Block 
A  of  Worksheet  04. 

1.  If  you  have  no  entries,  write  N/A  in  Block  A  and  go  on  to  Block 

B. 


2.  You  should  enter  the  first  step  number  through  the  last  step 
number  (e.g.,  1  through  7,  10  through  12). 


Block  B  is  used  when  a  single  step  must  be  performed  before  other 
steps  later  on  in  the  task. 


Purpose 

The  evaluator  will  use  this  information,  along  with  the  series  of 
steps  you  have  already  entered,  to  decide  whether  the  steps  of  the  task 
were  performed  in  order. 


Finding  Critical  Single  Steps 

Once  again,  you  should  use  the  criticality  questions  to  decide 
which  steps  should  be  performed  before  other  steps  later  in  the  task. 
You  should  go  through  the  task  and  ask  yourself,  "Will  any  of  the 
following  occur  if  this  step  is  not  performed  before  other  steps  later 
in  the  task?" 

1.  Injury  to  personnel. 

2.  Damage  or  loss  to  equipment  or  resources. 

3.  Performer  required  to  redo  the  task  or  part  of  the  task. 

4.  Co-worker  (working  with  the  performer  on  the  same  task) 
delayed  in  iFiitiating  or  completing  a  task  segment. 

REMEMBER  THAT  THE  ORDER  OF  TASK  PERFORMANCE  SHOULD  ONLY  BE 
EVALUATED  WHEN  IT  IS  CRITICAL  ACCORDING  TO  THE  CRITICALITY  QUESTIONS. 


Guidelines  for  Entering  Single  Steps 

In  Block  B,  enter  the  single  steps  that  must  be  performed  before 
other  steps  in  the  task. 

1.  If  you  have  no  entries  for  Block  B,  enter  N/A. 

2.  You  should  enter  the  single  step  number  and  the  numbers  of  the 
steps  it  should  be  performed  before,  in  the  following  format: 

a.  Step  3  before  Step  9. 

b.  Step  3  before  Steps  9  and  10. 

c.  Step  3  before  Steps  9,  10,  and  22. 

d.  Step  3  before  Steps  9  through  12. 


WORKSHEET  05:  EVALUATION  OF  END  PRODUCTS 


Overview 


Explanation 


One  of  the  most  obvious  ways  to  evaluate  a  performer  is  to  examine 
the  task  outcome  or  the  results  of  task  performance.  The  task 
end  product  is  the  outcome  or  result  that  the  task  is  aimed  at 
producing.  It  can  usually  be  observed  or  measured  when  the  task  is 
complete.  Evaluation  of  End  Products  is  the  evaluation  of  the  outcome 
or  result  of  task  performance. 


Purpose 

The  information  from  Worksheet  05  describes  for  the  evaluator: 

1.  Which  end  products  or  parts  of  end  products  should  be 
evaluated. 

2.  How  these  end  products  should  be  evaluated. 

3.  When  in  the  task  the  end  products  should  be  evaluated. 

The  evaluator  uses  this  information  to  decide  whether  or  not  the 
outcome  of  task  performance  is  acceptable. 

Completing  Worksheet  05 

The  following  information  will  be  entered  on  Worksheet  05: 

1.  Column  A  -  List  of  end  products  and/or  components  of  end 
products. 

2.  Column  B  -  Criteria  for  eva'uat'nq  end  products. 

3.  Column  C  -  Steps  associated  with  the  generation  of  end 
products. 


Examples  of  Worksheet  05 


10. 


Completed  examples  of  Worksheet  05  can  be  found  in  Figures  9  and 
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Worksheet  05;  Evaluation  of  End  Product 


Worksheet  05:  Evaluation  of  End  Product 


Worksheet  05:  Evaluation  of  End  Product 
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gear  inspection  |  Clean 


Worksheet  05:  Evaluation  of  End  Product 
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Figure  10.  Example  of  Completed  Worksheet  05; 
Preflight  KC-135A  (Continued) 
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Worksheet  06:  Evaluation  of  End  Product 


Ho  evidence  of  wear  and  damage 


Worksheet  05:  Evaluation  of  End  Product 


’.  M 
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Column  A  End  Products 


Explanation 


The  end  products  or  parts  of  end  products  of  task  performance  will 
be  listed  in  Column  A  of  Worksheet  05.  Instructions  will  be  given  for 
distinguishing  between  end  products  of  task  performance  and 
end  products  of  individual  steps  in  the  task. 


Purpose 

The  list  of  end  products  in  Column  A  tells  the  evaluator  which 
end  products  should  be  evaluated.  During  task  performance,  the 
evaluator  will  look  for  the  generation  of  those  end  products. 


Finding  Task  End  Products 

End  products  of  task  performance  can  be  generated  in  several 
different  ways,  depending  on  the  task  of  interest.  The  most  common 
ways  that  end  products  are  generated  are  as  follows; 

1.  One  end  product  can  be  generated  at  the  end  of  task 
performance.  For  instance,  the  main  end  product  of  the  LOX  servicing 
task  is  that  the  LOX  quantity  gages  read  full. 

2.  An  end  product  can  have  several  different  parts  or  components. 
These  parts  or  components  may  be  generated  at  different  steps  in  the 
task.  For  instance,  the  end  product  of  the  drag  chute  installation 
task  is  that  the  drag  chute  meets  certain  criteria  and  is  installed 
properly.  However,  different  aspects  of  proper  installation  are 
completed  at  different  steps  in  the  task. 

3.  A  task  can  have  several  different  end  products.  In  addition, 
some  of  these  end  products  may  have  several  parts  or  components.  The 
end  products  and  their  parts  or  components  may  be  generated  at 
different  steps  in  the  task.  For  instance,  the  preflight  task  has  many 
end  products.  This  is  true  of  most  tasks  which  involve  inspection  or 
cleaning. 

You  should  use  the  information  you  already  have  about  the  task  to 
find  the  task  end  products.  Some  information  you  may  find  useful  is 
described  below. 

Task  Objective.  Ask  yourself , "What  is  the  purpose  of  this  task; 
what  is  the  task  aimed  at  producing  or  accomplishing?"  The  answer  to 
this  question  may  be  a  task  end  product. 


Task  Title.  The  title  of  the  task  may  provide  information  about 
the  end  product  of  task  performance.  The  outcome  or  end  product  of 
task  performance  may  be  referred  to  in  the  task  title. 

Step  Descriptions.  The  words  in  the  individual  step  descriptions 
may  indicate  that  a  task  end  product  is  generated  at  that  step.  Look 
for  words  like  check,  clean,  ensure,  or  inspect.  These  words  often 
indicate  the  generation  of  task  end  products. 

Safety  Procedures.  Task  end  products  can  be  generated  when 
certain  steps  or  events  in  the  task  are  performed  for  later  safe 
operation  of  the  equipment.  (Task  end  products  are  usually  not 
generated  when  a  step  or  event  is  performed  solely  for  safety  during 
task  performance.  This  type  of  safety  procedure  will  be  included  on 
Worksheet  06.)  Examine  the  steps  of  the  task  that  are  performed  for 
safety  fol lowing  task  performance.  These  safety  events  may  result  in 
the  generation  of  important  task  end  products. 

Task  Outcome.  The  outcome  of  task  performance,  which  can  be 
observed  after  the  task  is  completed,  is  often  the  main  task 
end  product. 

Now  you  should  think  about  the  end  products  of  the  task. 


Qualifying  as  Task  End  Products 

In  order  to  qualify  as  task  end  products,  the  end  products  must 
meet  several  qualifications. 

First  of  all,  the  end  products  must  be  observable  and  measurable 
by  the  evaluator.  The  evaluator  must  be  able  to  see  the  end  product  at 
some  time  during  task  performance  or  when  the  task  is  complete. 

In  addition,  end  products  must  be  task  end  products  and  not  simply 
products  of  the  step  at  which  they  were  generated.  Avoid  confusing  the 
end  product  of  the  task  with  the  product  of  individual  steps  in  the 
task.  There  are  times  when  a  task  end  product  is  generated  at  almost 
every  step  in  the  task.  This  is  particularly  true  for  tasks  involving 
inspection  or  cleaning.  However,  many  tasks  have  only  one  end  product, 
which  is  usually  generated  towards  the  end  of  the  task. 

Some  tasks  have  only  one  end  product  with  several  different 
"parts."  These  "parts"  may  be  generated  at  different  points  in  the 
task.  For  instance,  in  the  drag  chute  installation  example  included  in 
this  handbook,  "Drag  Chute  Installed"  is  the  overall  end  product  of 
task  performance.  However,  the  precise  dimensions  and  orientation  of 
the  drag  chute  within  the  compartment  are  generated  early  in  the  task. 
Since  the  dimensions  and  orientation  of  the  drag  chute  are  critical  to 
successful  task  performance,  these  items  are  considered  parts  of  the 
overall  task  end  product. 


On  the  other  hand,  if  a  task  involved  opening  a  valve,  draining 
fluid  from  a  tank,  and  then  resealing  the  valve,  one  step  of  that  task 
might  result  in  an  open  valve.  However,  the  task  end  product  would  be 
that  the  tank  was  drained  and  the  valve  was  resealed. 

You  must  decide  whether  the  end  products  you  are  considering  are 
task  end  products  or  simply  products  of  the  step  at  which  they  occur. 
Only  end  products  of  the  task  should  be  evaluated. 

One  way  to  distinguish  between  step  and  task  end  products  is  to 
ask  yourself:  "WILL  THE  STATUS  AND/OR  FEATURES  OF  THE  END  PRODUCT 
GENERATED  AT  THIS  STEP  REMAIN  THE  SAME  AT  THE  END  OF  THE  TASK?"  If 
your  answer  is  "yes,"  then  the  product  you  are  considering  is  probably 
a  task  end  product. 

Based  on  these  qualifications,  you  should  decide  which  end 
products  or  parts  of  end  products  should  be  evaluated. 


Guidelines  for  Entering  End  Products 

Enter  the  end  products  or  parts  of  end  products  in  Column  A  of 
Worksheet  05.  Almost  every  task  should  have  at  least  one  end  product. 


Column  8  Criteria 


Explanation 


In  Column  B  you  will  describe  how  the  end  products  should  be 
evaluated.  Every  end  product  must  meet  certain  criteria  in  order  to  be 
acceptable.  For  each  end  product,  you  will  describe  those  criteria. 


Purpose 


The  evaluator  will  use  the  listed  criteria  to  tell  the  difference 
between  an  acceptable  and  a  unacceptable  end  product.  In  order  to 
conduct  fair  and  accurate  evaluations,  it  is  important  that  all 
evaluators  evaluate  the  end  products  according  to  the  same  criteria. 


Finding  Criteria 


The  evaluator  will  need  to  know  criteria  for  evaluation.  When  the 
end  product  must  meet  certain  criteria,  you  should  describe  those 
criteria. 


For  each  end  product  listed,  ask  yourself,  "Will  any  of  the 
following  occur  if  this  end  product  does  not  meet  certain  criteria?" 
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1.  Injury  to  personnel. 

2.  Damage  to  equipment  or  resources. 

3.  Performer  required  to  redo  the  task  or  part  of  the  task, 

4.  Co-worker  (working  with  the  performer  on  the  same  task) 
delayed  in  initiating  or  completing  a  task  segment. 

Some  types  of  criteria  you  may  want  to  consider  are  as  follows; 

1.  Function.  One  way  to  evaluate  an  end  product  is  to  see  if  it 
works  (functions).  For  example,  if  the  task  is  to  assemble  a 
component,  one  way  to  tell  if  it  is  properly  assembled  is  to  see  if  it 
works. 

2.  Dimensions.  Some  end  products  have  physical  dimensions  that 
are  important  (e.g.,  circumference,  length,  width,  height,  volume, 
weight).  For  example,  if  an  end  product  is  to  have  a  certain  width, 
then  width  may  be  an  important  feature  to  measure. 

3.  Other  physical  criteria.  Physical  criteria  include  color, 
texture,  consistency,  smoothness,  etc.  For  example,  if  an  end  product 
is  to  have  a  certain  color,  then  color  may  be  a  worthwhile  measure  to 
consider. 

4.  Accuracy  and  precision.  If  the  task  is  firing  a  weapon  at  a 
target,  then  it  might  be  important  to  measure  the  accuracy  and 
precision  of  the  performer  by  examining  the  target.  Accuracy  is  also 
important  to  evaluate  when  transmittal  of  information  (written  or 
verbal)  is  the  desired  result. 

5.  Status.  Some  end  products  must  be  in  a  specific  state  or 
condition  to  be  acceptable.  For  example,  it  might  be  important  for 
certain  switches  to  be  in  the  on  or  off  position. 

6.  Location.  To  be  acceptable,  end  products  frequently  must  be 
in  specific  locations.  For  example,  an  item  of  equipment  might  be 
stowed  in  a  particular  place. 

7.  Orientation.  Some  end  products  must  be  oriented  in  a  certain 
fashion  (in  a  certain  relationship  with  other  components).  For 
example,  an  equipment  item  might  need  to  be  in  an  upright  position  or  a 
certain  distance  from  another  piece  of  equipment. 

You  should  decide  what  type  of  criteria  should  be  evaluated  for 
each  end  product. 


REMEMBER,  ONLY  CRITERIA  THAT  ARE  CRITICAL  ACCORDING  TO  THE 
CRITICALITY  QUESTIONS  SHOULD  BE  EVALUATED. 


Describing  Criteria 

You  should  describe  at  least  one  type  of  criteria  for  each 
end-product  or  part  listed  in  Column  A.  When  describing  criteria  for; 

1.  Parts  of  end  products:  describe  one  or  more  types  of  criteria 
for  each  part  or  component  as  applicable. 

2.  One  end  product:  describe  one  or  more  types  of  criteria  for 
the  end  product.  If  you  have  only  one  end  product,  it  will  probably  be 
necessary  to  describe  several  types  of  criteria  (e.g.,  location, 
dimension,  function). 

3.  Multiple  end  products:  describe  one  or  more  types  of  criteria 
for  each  end  product  as  applicable. 

When  more  than  one  type  of  criteria  will  be  described  for  a 
particular  end  product  (e.g.,  location,  dimension),  these  criteria 
should  be  listed  in  separate  lines  of  the  worksheet.  This  will  make  it 
easier  when  you  assign  point  values  to  each  type  of  criteria  later  on 
in  the  TEF  development  procedures. 


Guidelines  for  Entering  Criteria 

The  criteria  should  be  entered  in  Column  B,  next  to  the  applicable 
end  product. 

1.  Be  sure  to  include  criteria  for  each  end  product  or  part. 

2.  Your  descriptions  should  be  written  in  terms  the  evaluator  can 
easi ly  understand. 

3.  Be  specific. 

4.  List  different  types  of  criteria  for  the  same  end  product  on 
separate  lines  of  the  worksheet. 

REMEMBER,  THE  EVALUATOR  WILL  USE  THESE  DESCRIPTIONS  TO  DISTINGUISH 
BETWEEN  AN  ACCEPTABLE  AND  AN  UNACCEPTABLE  END  PRODUCT. 

Some  examples  of  end  products  with  criteria  (from  various  tasks) 
are  shown  below; 


Drag  Chute  Installation  B-52D 


Drag  Chute  Control  Handle 


y^Drag  Chute 


y  Riser  Terminal 
y  Risers  and  Ripcord 
y'  Ripcord 

Drag  Chute  Door 

Crank  Handle 

Starter  Cartridge 


Operational,  moves  freely 
No  oil  or  moisture 

Dimensions:  15.75  inches  high,  21.50 
inches  wide,  and  36.50  inches  long 

Installed  in  compartment. 

Secured  and  locked  in  jettison  jaws. 

Pushed  up  and  out  of  way. 

Ball  terminal  of  ripcord  is  in 
terminal  fitting. 

Up  and  locked.  Red  indicator  is  in 
flush  with  aircraft  skin. 

Stowed  in  gunners  section. 


Starter  Cartridge  Removal  KC-135A 


Removed. 


y'  Breech  Cap  Inside  clean  and  clear. 

Secondary  Air  Inlet  Screen  Clean  and  clear, 

y'  Blowout  Plug  Clean  and  clear. 

y/  Locking  Lugs  Clean  and  clear. 

/ 

y'  Firing  Pin  Clean  and  clear. 

y^Breech  Lock,  Electrical  Outlet  Clean  and  clear. 

/ 

y Seats  Clean  and  clear. 


Clean  and  clear. 
Clean  and  clear. 


Clean  and  clear. 


Clean  and  clear. 


Breech  Cap 


Filler  Cap  Dipstick 


Clean  and  clear. 

Stored  in  plastic  bag  with  silica 
gel . 


Oi 1  Servicing  FB-1 1 lA 


Area  clean. 


Installed,  locking  lugs  engaged. 
Serviced  to  specs. 

Closed  and  aligned. 


Access  Doors 


Water  Tank 


Water  Tank  Cap 


Water  Servicing  to  FB-lllA 


Full. 


Installed  securely. 


*Note  that  for  some  end  products,  more  than  one  type  of  criteria  have 
been  specified. 


Column  C  Steps 


Explanation 

Most  end  products  are  associated  with  a  particular  step  in  the 
task.  There  are  times  when  task  end  products  are  actually  generated 
prior  to  the  end  of  the  task.  When  this  occurs,  you  should  determine 
which  step  is  associated  with  each  end  product.  In  addition,  certain 
end  products  cannot  be  easily  observed  and  evaluated  when  the  task  is 
complete.  You  will  identify  those  end  products  for  the  evaluator. 


Purpose 


This  information  will  tell  the  evaluator  when  to  watch  for  the 
generation  of  end  products. 


Finding  Steps 

Think  about  the  end  products  you  have  listed, 
is  associated  with  each  end  product. 


Decide  which  step 


You  should  also  decide  which  end  products  cannot  be  evaluated  when 
the  task  is  complete.  Some  examples  of  end  products  that  must  be 
evaluated  during  task  performance  are  indicated  by  checks  (y)  in  the 
previous  examples. 


Guidelines  for  Entering  Steos 


Enter  the  step  associated  with  each  end  product  in  Column  C. 

Enter  a  check  in  Column  A  next  to  those  end  products  which 

must  be  evaluated  during  task  performance. 


YOUR  ARE  NOW  READY  TO  GO  ON  TO  WORKSHEET  06. 


WORKSHEET  06:  EVALUATION  OF  SAFETY  PROCEDURES  AND  REGULATIONS 


I 

Overview 


Explanation 

Often  certain  procedures  and  regulations  are  followed  in  order  to 
protect  equipment  and  people  during  task  performance.  Following  these 
safety  procedures  and  regulations  can  be  an  important  part  of  task 
performance.  When  certain  safety  procedures  or  regulations  are  not 
followed,  damage  to  equipment  or  injury  to  personnel  in  the  surrounding 
areas  could  occur.  One  of  the  indications  of  successful  task 
performance  may  be  whether  the  performer  followed  safety  procedures  and 
regulations  during  task  performance. 


Purpose 

The  information  on  Worksheet  06  will  describe  for  the  evaluator: 

1.  The  safety  procedures  and  regulations  which  should  be 
evaluated. 

2.  When  to  evaluate  following  safety  procedures  and  regulations. 

The  evaluator  uses  this  information  to  decide  whether  the 
appropriate  safety  procedures  and  regulations  were  followed. 

Completing  Worksheet  06 

To  complete  Worksheet  06,  you  will  enter: 

1.  Column  A  Safety  procedures  and  regulations. 

2.  Column  B  Steps. 


Examples  of  Worksheet  06 

Examples  of  completed  Worksheet  06  are  shown  in  Figures  11  and 


Worksheet  06:  Evaluation  of  Safety  Procedures  and  Regulations 


Personnel  clear  of  vent.  Performer  stands  to  one  side 


Worksheet  06;  Evaluation  of  Safety  Procedures  and  Regulations 


Worksheet  06:  Evaluation  of  Safety  Procedures  and  Regulations 


Worksheet  06:  Evaluation  of  Safety  Procedures  and  Regulations 


In  Column  A  you  should  enter  a  description  of  the  safety 
procedures  and  regulations  that  should  be  evaluated. 


Purpose 


This  information  describes  for  the  evaluator  the  performer  actions 
that  reflect  the  following  of  safety  procedures  and  regulations.  The 
evaluator  will  use  this  description  to  decide  whether  the  safety 
procedures  and  regulations  were  correctly  performed. 


Finding  Safety  Procedures  and  Regulations 

1.  Some  procedures  and  regulations  can  be  found  by  examining  the 
step  descriptions.  Words  like  monitor,  ensure,  or  check  often  indicate 
a  safety  procedure  that  is  performed  for  safety  during  task 
performance. 

2.  Often,  certain  handles,  controls,  switches,  valves,  etc.  must 
be  in  certain  positions  in  order  to  ensure  safety  during  task 
performance. 

3.  There  are  times  when  certain  switches,  handles,  nozzles,  etc. 
must  be  moved  or  rotated  in  a  certain  manner  (e. g.,  slowly,  steadily, 
guickly)  for  safety  during  task  performance. 

4.  Certain  items,  such  as  plugs,  valves,  or  nozzles,  must  be 
clean  and  clear  to  ensure  safety  during  task  performance. 

5.  In  addition  to  safety  procedures  and  regulations  that  are 
specific  to  the  task  under  consideration,  there  are  often  general 
safety  procedures  and  regulations  which  must  be  followed. 


Which  Safety  Procedures  and  Regulations  Should  be  Evaluated 

You  should  use  the  criticality  questions  listed  below  to  help  you 
decide  which  safety  procedures  and  regulations  should  be  evaluated.  Go 
through  the  task  and  ask  yourself,  "Will  any  of  the  following  occur  if 
specified  procedures  and  regulations  are  not  followed?" 


1.  Injury  to  personnel. 


2.  Damage  to  equipment  or  resources. 

3.  Performer  required  to  redo  the  task  or  part  of  the  task. 


4.  Co-worker  (working  with  the  performer  on  the  same  task) 
delayed  in  initiating  or  completing  a  task  segment. 

Steps  or  events  that  are  done  only  for  safety  following  task 
performance  should  not  be  considered  now.  For  example,  during  an 
aircraft  maintenance  task,  some  safety  procedures  are  performed  to 
protect  personnel  during  task  performance,  while  others  are  performed 
to  guarantee  the  safety  of  crew  members  during  flight.  The  safety 
procedures  that  are  concerned  with  later  safe  operation  of  the 
equipment  will  probably  be  considered  end  products.  Safpty  procedures 
concerned  with  the  immediate  safe  performance  of  the  task  at  hand 
should  be  considered  here.  The  use  of  protective  equipment,  including 
clothing,  should  be  considered  a  safety  procedure. 

REMEMBER,  SAFETY  SHOULD  ONLY  BE  EVALUATED  WHEN  IT  IS  CRITICAL 
ACCORDING  TO  THE  CRITICALITY  QUESTIONS, 

AND 

YOU  SHOULD  ONLY  INCLUDE  SAFETY  PROCEDURES  AND  REGULATIONS  THAT  ARE 
PERFORMED  FOR  SAFETY  DURING  TASK  PERFORMANCE. 


Guidelines  for  Entering  Safety  Procedures  and  Regulations 

Enter  the  safety  procedures  and  regulations  in  Column  A  of 
Worksheet  06. 

1.  You  should  describe  the  safety  procedures  and  regulations  in 
terms  the  evaluator  will  understand.  Common,  well-known  procedures  and 
regulations  can  be  referred  to  by  name  only. 

2.  When  you  are  describing  a  safety  procedure  that  is  not  common 
or  is  specific  to  the  task  under  consideration,  you  should  describe 
performer  actions  or  outcomes  that  the  evaluator  can  see. 

3.  If  you  have  no  entries,  enter  N/A  in  Column  A,  and  go  onto 
Worksheet  07. 

REMEMBER,  THE  EVALUATOR  WILL  USE  YOUR  DESCRIPTIONS  TO  DECIDE 
WHETHER  THE  SAFETY  PROCEDURES  OR  REGULATIONS  WERE  PERFORMED  CORRECTLY. 

Acceptable  entries  should  resemble  the  following  examples; 
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General  Safety  Procedures  and  Re 


Aircraft  grounded. 

Wear  protective  clothing. 

External  power  disconnected. 

Fire  extinguisher  within  easy  reach. 

Personnel  clear  of  vents,  carts,  etc. 

Clearance  obtained. 

ecific  Safety  Procedures  and  Requlations  From  Several  Tasks 


Seat  pins  installed. 

Ensure  that  fuel  is  adequately  distributed. 

Fuel  control  switch  in  feed  position. 

All  eight  lights  of  fire  detection  system  illuminated. 
Manifold  in  open  position. 

Throttles  closed. 

Brakes  set  and  chocked. 

Landing  gear  handle  down  and  locked. 

Visually  check  switches. 

Monitor  indicator  lights. 

Open  nozzle  slowly. 

Do  not  build  pressure  over  30+  5  psi. 


Column  B  Steps 


Explanation 

In  Column  B,  you  should  list  the  steps  when  safety  procedures  or 
regulations  should  be  evaluated.  Steps  should  be  listed  for  each  entry 
in  Column  A. 


Purpose 

The  list  of  steps  tells  the  evaluator  when  to  evaluate  the  safety 
procedures  and  regulations. 


Finding  Steps 

Safety  procedures  and  regulations  can  be  evaluated  for: 

1.  A  step  in  the  task. 

2.  Several  different  steps  in  toe  task. 

3.  A  series  of  steps  in  the  task. 

4.  The  whole  task . 


Once  again  you  may  want  to  refer  to  the  criticality  questions  to 
help  you  decide  when  to  evaluate  each  safety  procedure  and  regulation. 
Remember,  for  each  safety  procedure  or  regulation  listed  in  Column  A, 
you  should  go  through  the  steps  and  ask  yourself,  "Will  any  of  the 
following  occur  if  this  safety  procedure  or  regulation  is  not  followed 
at  this  step?" 

1.  Injury  to  personnel. 

2.  Damage  to  equipment  or  resources. 

3.  Performer  required  to  redo  the  task  or  part  of  the  task. 

4.  Co-worker  (working  with  the  performer  on  the  same  task) 
delayed  in  initiating  or  completing  a  task  segment. 


Guidel i nes  for  Entering  Steps 


WORKSHEET  07:  EVALUATION  OF  TOOLS,  EQUIPMENT,  AND  MATERIALS  USE 


Overview 


Explanation 

Most  tasks  are  performed  with  the  aid  of  tools,  equipment,  and 
materials.  There  are  times  when  successful  task  performance  could  be 
prevented  if  misuse  of  tools,  equipment,  or  materials  occurs.  On 
Worksheet  07  you  will  enter  information  about  how  the  tools,  equipment, 
and  materials  should  be  used  during  task  performance. 


Purpose 

The  information  on  Worksheet  07  tells  the  evaluator: 

1.  Which  tools,  equipment,  and  materials  have  uses  that  should  be 
evaluated. 

2.  The  specific  size  or  type  of  tools,  equipment,  and  materials 
that  should  be  used. 

3.  The  correct  use  of  the  tools,  equipment,  and  materials. 

4.  When  in  the  task,  the  use  of  each  item  of  tools,  equipment,  or 
material  should  be  evaluated. 

The  evaluator  uses  the  information  to  decide  whether  the  items 
were  used  correctly. 


Completing  Worksheet  07 

In  order  to  complete  Worksheet  07,  you  will  enter  the  following 
informat  ion: 


1. 

Co  1 umn 

A 

Tools,  Equipment,  and 

Materials 

2. 

Column 

B 

Size  or  Type 

3. 

Column 

C 

Correct  Use 

4. 

Column 

0 

Steps  Associated  With 

Use 

Examples  of  Worksheet  07 


See  Figures  13  and  14  for  completed  examples  of  Worksheet  07. 


Worksheet  07:  Evaluatton  of  Tools,  Equipment,  and  Materials  Use 


Worksheet  07:  Evaluation  of  Tools,  Equipment,  and  Materials  Use 


Figure  14.  Example  of  Completed  Worksheet  07: 
Preflight  KC-135A. 
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Several  different  items  of  tools,  equipment,  and  materials  are 
normally  used  to  complete  a  task.  In  Column  A,  you  will  enter  those 
items  which  have  uses  that  should  be  evaluated. 


Purpose 

This  list  will  tell  the  evaluator  which  tools,  equipment,  and 
materials  should  be  evaluated.  It  is  important  that  all  evaluators 
evaluate  the  use  of  the  same  items. 


Finding  Critical  Tools,  Equipment,  and  Materials 

A  list  of  example  tools,  equipment,  and  materials  is  shown  below. 
This  list  is  not  complete,  but  it  should  give  you  an  idea  of  the  kinds 
of  items  to  consider. 

Tools,  Equipment,  and  Materials 


Communications  Equipment 
Headsets 

Handheld  transmitters 

Receivers 

Radios 

Whistles 


Test  Equipment 

Meters  (e.g.,  multimeter) 

Gages 

Specialized  test  sets 


Safety  Equipment 
Chocks 

Fire  extinguisher 
Engine  run  fence 
Fire  bottle 


Tools 
Mini-kit 
Hand  crank 
Tool  box 
Ground  cord 
Torque  wrench 
Cowling  wrench 
Tire  gage 
Lock  wire 
Wire  brush 
Screwdri ver 
Sump  drain  tool 


Materials 

Rags 

Fluids  (e.g.,  hydraulic) 
Sump  bags 

Solvents  or  cleaners  (e.g., 
freon) 

Gels  (e.g.,  silica) 

Duct  tape 
Scotch  brite 
Window  cleaner 


(Continued) 


Miscellaneous 

Wands 

Marshal ler's  vest 
B-1/B-4/B-5  stand 
Power  unit/air  cart 
Stands 
Dipstick 

Waste  fuel  containers 

Drip  pans 

Extra  ground  wire 

Drain  hose 

Pogos 


Keep  in  mind  that  the  operational  equipment  (e.g.,  aircraft  or 
other  equipment  being  worked  on)  is  not  categorized  as  tools, 
equipment,  or  materials  here.  When  the  task  involves  repairing  or 
maintaining,  cleaning,  or  inspecting  an  item  of  equipment,  that  item 
would  not  be  considered  tools,  equipment,  or  material  for  the  purposes 
of  TEF  development.  Only  those  items  used  to  repair,  maintain,  clean, 
or  inspect  the  operational  equipment  or  other  equipment  being  worked  on 
will  be  listed  on  this  worksheet. 

Think  about  the  tools,  equipment,  and  materials  required  to 
complete  the  task. 


Which  Tools,  Equipment,  and  Materials  Should  be  Evaluated 

You  should  decide  which  tools,  equipment,  and  materials  should  be 
evaluated  by  applying  the  criticality  questions.  For  each  item  used  in 
the  task,  ask  yourself,  "Will  any  of  the  following  result  if  misuse  of 
this  tool,  equipment,  or  material  occurs?" 

1.  Injury  to  personnel. 

2.  Damage  or  loss  to  equipment  or  resources. 

3.  Performer  required  to  redo  the  task  or  part  of  the  task. 

4.  Co-worker  (working  with  the  performer  on  the  same  task) 
delayed  in  initiating  or  completing  a  task  segment. 

Keep  in  mind  that  misuse  of  tools,  equipment,  and  materials 
includes  the  following: 

1.  Using  the  tools,  equipment,  or  materials  in  an  incorrect 
manner. 


2.  Using  the  wrong  tool,  equipment,  or  material. 


REMEMBER,  THE  USE  OF  TOOLS,  EQUIPMENT,  AND  MATERIALS  SHOULD  ONLY 
BE  EVALUATED  WHEN  IT  IS  CRITICAL  ACCORDING  TO  THE  CRITICALITY 
QUESTIONS. 


Guidelines  for  Entering  Tools.  Equioment.  and  Materials 


The  tools,  equipment,  and  materials  should  be  listed  in  Column  A 
of  Worksheet  07. 

If  you  have  no  entries  for  tools,  equipment,  and  materials,  go 
on  to  Worksheet  08. 


Column  B  Size/Type 


Explanation 


There  are  times  when  only  specific  sizes  or  types  of  tools, 
equipment,  or  materials  should  be  used.  When  this  occurs,  the 
appropriate  size  or  type  of  each  tool,  equipment  item,  or  material  will 
be  entered  in  Column  B  of  Worksheet  07. 


Purpose 


The  information  in  this  column  provides  the  evaluator  with  a 
specific  description  of  the  tools,  equipment,  or  materials  to  be  used. 
The  evaluator  can  check  the  size  or  type  entered  in  Column  B  against 
the  size  or  type  of  item  the  performer  actually  uses.  By  doing  this 
the  evaluator  can  decide  whether  or  not  the  performer  is  using  the 
correct  tools,  equipment,  or  materials. 


Guidelines  for  Entering  Size/T 


Enter  the  size  or  type  in  Column  B  of  Worksheet  07. 

1.  It  may  not  be  necessary  to  enter  a  size  or  type  for  every  item 
listed  in  Column  A.  It  is  OK  to  enter  "Standard"  when  size/type  is  not 
important.  For  every  entry  in  Column  A,  complete  Column  B  as  follows: 

a.  Enter  the  description  of  the  size  or  type 


b.  Enter  "Standard" 


2.  Be  sure  to  make  the  entry  in  the  same  row  as  the  tool, 
equipment  item,  or  material  to  which  the  entry  refers. 


Some  examples  of  tools,  equipment, 
size/type  are: 

Tools,  Equipment, 
and  Materials 

Rags 

Lockwire 

Wrench 

Freon  cleaner 
Drip  pans 
Screwdriver 
Wire  brush 


and  materials  and  their 


Size/Type 

Clean,  dry 

MS20995C32 

Cowling 

Standard 

Clean,  metal 

Phillips 

Standard 


Column  C  Correct  Use 


Explanation 

In  Column  C  you  will  describe  exactly  how  the  tools,  equipment, 
and  materials  should  be  used. 


Purpose 

The  information  in  Column  C  describes  for  the  evaluator  exactly 
what  performer  actions  represent  acceptable  use  of  tools,  equipment, 
and  materials.  The  evaluator  uses  this  information  to  decide  whether 
the  performer  used  the  tools,  equipment,  or  materials  in  the  correct 
manner. 


Guidelines  for  Entering  Correct  Use 

The  correct  use  should  be  entered  in  Column  C. 

1.  Describe  a  correct  use  for  every  item  listed  in  Column  A. 

2.  Describe  how  the  items  are  used. 

3.  Be  specific. 

4.  Describe  observable  performer  actions. 

5.  When  a  particular  item  is  used  for  more  than  one  purpose,  be 
sure  to  make  separate  entries  for  each  different  use. 

REMEMBER  THE  EVALUATOR  WILL  USE  YOUR  DESCRIPTIONS  TO  DECIDE 
WHETHER  THE  ITEMS  WERE  USED  CORRECTLY. 


Some  examples  of  acceptable  descriptions  of  correct  use  are  shown 


below: 

Tools,  Equipment, 
and  Materials 


Size/Type 


Correct  Use 


Rags 

Clean,  dry 

Use  to  clean  area  around 
overboard  vent. 

Lockwi re 

MS20995C32 

Pass  through  center  of  plug. 

Wrench 

Cowl i ng 

Open  cowling. 

Freon  cleaner 

Standard 

Clean  electrical  contact  pin. 

Drip  pans 

Clean,  metal 

Place  under  overboard  vent. 

Screwdriver 

Phi  1 1 ips 

Rotate  fasteners  1/2  turn  CCW. 

Wirebrush 

Standard 

Clean  inside  breech  and  cap. 

Column  0  Stec 


Explanation 

You  have  already  listed  the  tools,  equipment,  and  materials,  the 
appropriate  size  or  type,  and  the  correct  use.  In  Column  D  you  will 
list  the  steps  when  these  items  should  be  evaluated. 


Purpose 


This  list  tells  the  evaluator  exactly  when  in  the  task  that  the 
use  of  tools,  equipment,  and  materials  should  be  evaluated. 


Finding  Critical  Steps 

The  use  of  tools,  equipment,  and  materials  can  be  evaluated  for: 

1.  A  step  in  the  task. 

2.  Several  steps  in  the  task. 

3.  A  sequence  of  steps  in  the  task. 

4.  The  whole  task. 


L«T*1 


Once  again,  you  may  want  to  apply  the  criticality  questions  to 
help  you  decide  when  the  tools,  equipment,  and  material  uses  you  have 
listed  should  be  evaluated. 

For  each  item  listed  in  Column  A,  you  should  go  through  the  task 
and  ask  yourself,  "Will  any  of  the  following  result  if  misuse  of  this 
item  occurs  at  this  step?" 

1.  Injury  to  personnel. 

2.  Damage  or  loss  to  equipment  or  resources. 

3.  Performer  required  to  redo  the  task  or  part  of  the  task. 

4.  Co-worker  (working  with  the  performer  on  the  same  task) 
delayed  in  initiating  or  completing  a  task  segment. 

REMEMBER,  THE  TOOLS,  EQUIPMENT,  AND  MATERIALS  USE  SHOULD  ONLY  BE 
EVALUATED  WHEN  IT  IS  CRITICAL  ACCORDING  TO  THE  CRITICALITY  QUESTIONS. 


Guidelines  for  Enterinq  Steps 


The  steps  should  be  entered  in  Column  D. 

1.  Be  sure  to  enter  the  steps  in  the  correct  row  of  the 
worksheet.  For  example,  when  a  particular  item  has  different  uses,  be 
sure  to  enter  the  steps  in  the  row  next  to  the  appropriate  use. 


item. 


2.  Enter  all  of  the  steps  that  should  be  evaluated  for  each 


YOU  ARE  NOW  READY  TO  GO  ON  TO  WORKSHEET  08. 


WORKSHEET  08:  EVALUATION  SCENARIO 


Overview 


Explanation 


Worksheet  08  is  used  to  describe  how  the  task  should  be  presented 
to  the  performer  for  evaluation  purposes.  When  Worksheet  08  is 
completed,  an  evaluation  scenario  results. 


Purpose 


The  evaluator  will  use  the  information  from  Worksheet  08  to  set  up 
the  task  for  the  evaluation.  The  instructions  for  setting  up  the 
evaluation  scenario  are  provided  for  the  evaluator  to  ensure  that 
different  evaluators  set  up  the  task  in  the  same  way.  In  order  to 
conduct  fair  evaluations,  it  is  important  that  each  performer  is 
presented  with  the  same  evaluation  scenario  and  that  different 
evaluators  use  the  same  scenario  to  evaluate  the  same  task. 


Completing  Worksheet  08 


In  order  to  complete  the  worksheet,  you  will  describe: 

1.  Line  A  Best  Evaluation  Method. 

2.  Block  B  Preventative  Environmental  Conditions. 

3.  Block  C  Presentation  of  Operational  Equipment. 

4.  Block  0  Help  Permitted. 

5.  Block  E  Presentation  of  Performer  Tools,  Equipment,  and 

Materials. 

6.  Block  F  Evaluator  Equipment. 

7.  Block  G  Evaluator  Time  Estimates. 

8.  Block  H  Number  of  Evaluators. 


Examples  of  Worksheet  08 


Examples  of  completed  Worksheet  08  are  shown  in  Figures  15  and 
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Figure  16. 


Example  of  Completed  Worksheet  08: 
Preflight  KC-135A. 
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Block  A  Evaluation  Method 


Explanation 

Now  that  you  have  decided  what  should  be  evaluated  during  task 
performance,  you  can  select  a  method  of  evaluation.  Three  evaluation 
methods  were  discussed  earlier  in  this  handbook.  You  reviewed  these 
three  methods  to  determine  if  any  of  the  three  could  be  used  to 
evaluate  the  task  under  consideration.  Now  you  will  select  the  best 
met  hod. 


Purpose 


Block  A  tells  the  evaluator  which  evaluation  method  should  be  used 
to  evaluate  the  task. 


Selecting  the  Best  Method 


The  three  evaluation  methods  are  as  follows: 

1.  Method  1.  Use  Actual  Equipment  in  Job  Environment.  In  this 
method,  task  performance  is  evaluated  as  it  actually  occurs  in  the  job 
environment. 

2.  Method  2.  Use  "Rigged"  Actual  Equipment.  In  this  method, 
task  performance  is  evaluated  on  actual  equipment  that  has  been 
"rigged"  to  create  the  desired  situation. 

3.  Method  3.  Use  Trainer  or  Simulator.  In  this  method,  task 
performance  is  evaluated  on  a  trainer  or  simulator. 

To  select  an  evaluation  method,  ask  yourself  the  following 
questions: 

Questions  on  Method  1 

1.  Is  the  task  under  consideration  performed  frequently  in  the 
job  environment? 

NO  -  Go  to  Method  2. 

YES  -  Go  to  the  next  question. 

2.  Will  the  evaluation  of  task  performance  interfere  with 
successful  completion  of  the  task? 


NO  -  Method  1  will  probably  be  appropriate.  Think  about  the 
critical  aspects  of  the  task  to  be  sure  they  can  be  evaluated  during 
actual  task  performance. 

YES  -  Go  to  Method  2. 

Questions  on  Method  2 

1.  Is  it  possible  to  "rig"  the  operational  equipment  to  present 
the  desired  task? 

NO  -  Go  to  Method  3. 

YES  -  Go  to  the  next  question. 

2.  Is  it  possible  to  "rig"  the  aircraft  without  creating  a 
hazardous  situation? 

NO  -  Go  to  Method  3. 

YES  -  Method  2  will  probably  be  appropriate.  Think  about  the 
critical  aspects  of  task  performance  to  be  sure  they  can  be  evaluated 
using  "rigged"  equipment. 

Questions  on  Method  3 


1.  Is  a  trainer  or  simulator  available? 

NO  -  Method  1  or  Method  2  must  be  used.  Reconsider  the 
questions  above. 

YES  -  Go  to  the  next  question. 

2.  Can  the  task  be  presented  on  the  trainer  or  simulator? 

NO  -  Method  1  or  Method  2  must  be  used.  Reconsider  the 
questions  above. 

YES  -  Method  3  will  probably  be  appropriate.  Think  about  the 
critical  aspects  of  task  performance  to  be  sure  they  can  be  evaluated 
using  the  trainer  or  simulator. 


Guidelines  for  Entering  Evaluation  Method 

Based  on  the  questions  above,  you  have  selected  an  evaluation 
method.  Record  your  choice  in  Line  A  of  Worksheet  08.  Enter  one  of 
the  following: 


1.  Actual  Equipment,  Job  Environment 

2.  Actual  Equipment,  Rigged 

3.  Trainer  (and  name  of  trainer) 


Block  B  Preventative  Environmental  Conditions 


Explanation 

In  Block  B  you  will  enter  the  preventative  environmental 
conditions.  Preventative  environmental  conditions  are  conditions  under 
which  the  task  should  not  be  performed  or  evaluated.  These  include 
conditions  under  which  it  would  be  unfair,  impractical,  hazardous,  or 
impossible  to  conduct  the  evaluation. 


Purpose 

The  evaluator  will  use  this  information  to  select  an  appropriate 
and  fair  time  to  conduct  the  evaluation. 


Findinq  Preventative  Environmental  Conditions 


The  most  common  way  environmental  conditions  prevent  task 
performance  or  evaluation  is  by  creating  a  hazardous  situation  (e.g., 
lightning  during  LOX  servicing).  Another  way  environmental  conditions 
prevent  task  performance  is  by  making  the  task  extremely  difficult  to 
perform  and  thus  unfair  to  evaluate  (e.g.,  icy  surfaces  during  the 
towing  task).  The  evaluation  must  be  conducted  when  no  preventative 
conditions  exist.  You  should  select  those  conditions  which  would 
prevent  the  task  from  being  performed  or  evaluated. 

Think  about  the  following  list  of  general  environmental 
conditions.  For  each  general  environmental  condition,  ask  yourself, 
"Are  there  any  specific  conditions  under  which  the  evaluation  should  not 
be  performed?" 

1.  Light. 

2.  Location. 

3.  Weather. 

4.  Noise. 

5.  Temperature. 


Guidelines  for  Entering  Preventative  Environmental  Conditions 


Enter  the  specific  preventative  environmental  conditions  in  Block 
B  of  Worksheet  08.  If  there  are  no  preventative  environmental 
conditions,  write  N/A  in  Block  B, 

Some  entries  you  may  want  to  consider  are  shown  below: 


1.  Light: 


Inadequate  light. 


2.  Location: 


3.  Weather: 


4.  Noise: 


Aircraft  within  50  feet  of  hangar. 
Aircraft  within  50  feet  of  buildings. 
Aircraft  within  50  feet  of  other  aircraft 
Aircraft  in  dock. 

Icy  surface. 

Unlevel  surface. 

Rain. 

Snow. 

Lightning  within  3  miles. 

High  winds. 

Flightline  level. 

Noise  beyond  correction. 

Noise  requiring  ear  defenders. 


5.  Temperature:  Above  90“F. 

Below  32‘*F. 

Chi  1 1  factor  too  low. 


Block  C  Presentation  of  Operational  Equipment  or  Task 


Explanation 

This  is  the  most  important  part  of  the  evaluation  scenario.  In 
Block  C,  you  should  describe  how  the  operational  equipment  should  be 
presented  to  the  performer. 


Purpose 

The  evaluator  will  follow  your  description  to  prepare  the 
operational  equipment  for  the  evaluation.  It  is  important  that  all 
evaluators  set  up  the  operational  equipment  in  the  same  manner  so  that 
performers  can  be  evaluated  in  the  same  way. 


Guidelines  for  Entering  Presentation  of  Operational  Equipment/Task 


You  should  be  very  specific  and  describe  exactly  what  must  be  done 
to  prepare  the  operational  equipment  for  the  evaluation. 

You  can  use  the  questions  below  to  help  you  describe  how  to 
prepare  the  operational  equipment.  You  do  not  have  to  respond  to  every 
question,  but  you  should  provide  all  the  information  the  evaluator  will 
need  to  prepare  the  operational  equipment. 


REMEMBER,  THE  EVALUATOR  WILL  FOLLOW  YOUR  DESCRIPTION  TO  SET  UP  THE 
OPERATIONAL  EQUIPMENT.  IN  ORDER  TO  CONDUCT  FAIR  EVALUATIONS,  IT  IS 
IMPORTANT  THAT  ALL  EVALUATORS  SET  UP  THE  OPERATIONAL  EQUIPMENT  IN  THE 
SAME  MANNER. 

Question  1.  Should  the  operational  equipment  require  the  task  to 
be  performed?  ^For  example, 

1.  The  aircraft  must  require  LOX  servicing. 

2.  The  aircraft  must  require  refueling. 

3.  The  aircraft  must  require  starter  cartridge  removal. 

4.  The  aircraft  must  require  towing. 

If  your  answer  is  “yes,'*  enter:  the  name  of  the  equipment,  the 
words  "must  require,"  and  the  name  of  the  task  in  Block  C. 

Question  2.  What  general  conditions  should  exist  when  the  task  is 
performed?  For  instance, 

1.  What  must  be  turned  on? 

2.  What  must  be  turned  off? 

3.  What  should  not  be  in  the  surrounding  area? 

4.  Should  the  equipment  be  grounded? 

5.  What  conditions  would  result  in  safety  hazards? 

Some  examples  of  entries  are: 

1.  Power  on. 

2.  Power  off. 

3.  No  other  aircraft  within  50  feet. 

4.  Equipment  grounded. 

5.  No  other  maintenance  being  performed. 

Enter  the  conditions  in  Block  C. 

Question  3.  What  should  be  pre-performed  or  set  up  for  the 
performer  before  the  start  of  the  evaluation?  For  instance, 

1.  What  preliminary  steps  of  the  task  are  not  part  of  the 
evaluation? 
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2.  What  access  doors  should  be  open? 

3.  What  obstructing  components  should  be  removed? 

4.  What  must  be  connected  or  disconnected? 

Some  examples  of  entries  are: 

1.  Steps  1  to  3  pre-performed. 

2.  Access  doors  (X3)  open. 

3.  Cowling  removed. 

4.  Relay  N2  disconnected. 

Enter  the  details  in  Block  C. 

Question  4.  Are  there  any  other  specific  "rigging"  details?  For 
instance, 

1.  What  dials,  valves,  controls,  or  switches  should  be 
positioned? 

2.  What  adjustments  should  be  made? 

Some  examples  of  entries  are: 

1.  Dial  X7  positioned  at  3. 

2.  Switch  9  off. 

Enter  the  details  in  Block  C. 

Question  5.  What  faults,  malfunctions,  or  defective  parts  should 
be  inserted?  Tlhis  question  does  not  apply  when  Evaluation  Method  1  is 
used;  however,  when  a  trainer  or  simulator  is  used,  you  should  provide 
a  response  to  this  question.) 

Some  examples  of  entries  of  malfunctions  are: 

1.  Hot  start  malfunction. 

2.  Loss  of  aircraft  power,  no  external  power  available. 

3.  No  fuel  flow,  other  indications  normal. 

4.  Standby  pump  failure. 

5.  Low  oil  pressure  light  on. 

Enter  the  faults,  malfunctions,  etc.  in  Block  C. 
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Review  all  of  the  entries  in  Block  C.  Ask  yourself  the 
following: 

1.  Have  I  provided  all  the  details  the  evaluator  will  need  to 
set  up  the  evaluation? 

2.  Are  any  of  the  instructions  too  vague  or  unclear?  Will  the 
evaluator  be  able  to  use  these  instructions  to  set  up  the  operational 
equipment? 

Add  to  or  change  your  entries  as  necessary. 


Block  D  Help  Permitted 


Explanation 


There  are  times  when  the  performer  is  allowed  assistance  during 
task  performance.  In  Block  D,  you  will  describe  when  and  what  type  of 
help  is  permitted  during  the  performance  of  the  task. 


Purpose 

The  information  in  Block  0  tells  the  evaluator  when  and  what  type 
of  assistance  the  performer  is  allowed.  In  order  to  conduct  fair 
evaluations,  it  is  important  that  all  performers  receive  the  same  type 
and  amount  of  help. 


Guidelines  for  Entering  Help  Permitted 

Describe  when  help  is  allowed  and  what  type  of  help  is  allowed  in 
Block  D. 

1,  Enter  the  parts  of  the  task  that  require  more  than  one 
person. 

2.  Enter  the  parts  of  the  task  where  the  performer  is  allowed  to 
request  assistance. 

REMEMBER,  THE  EVALUATOR  WILL  USE  YOUR  DESCRIPTION  TO  DECIDE  WHEN 
AND  WHAT  TYPE  OF  ASSISTANCE  IS  PERMITTED. 


Some  examples  of  help  permitted  are  shown  below: 

1.  Other  members  of  the  team  will  perform  their  duties. 

2.  Performer  can  request  assistance  at  Steps  7  to  10. 


3.  Performer  can  request  responses  from  ground  observer. 


4.  Performer  can  receive  assistance  in  connecting  external 
power. 

5.  No  help  is  allowed. 


Block  E  Presentation  of  Performer  Tools,  Equipment,  and  Materials 


Explanation 

You  have  already  described  how  the  operational  equipment  should  be 
presented  to  the  performer.  Now  you  will  describe  how  the  tools, 
equipment,  and  materials  needed  to  perform  the  task  should  be 
presented. 


Purpose 

The  evaluator  will  use  your  descriptions  to  prepare  the  tools, 
equipment,  and  materials  for  the  evaluation.  It  is  important  that  the 
tools,  equipment,  and  materials  are  presented  to  each  performer  in 
exactly  the  same  way. 


Guidelines  for  Entering  Performer  Tools,  Equipment,  and  Materials 

In  Block  D  of  Worksheet  08  you  should  list  the  tools,  equipment, 
and  materials  the  performer  will  need  to  complete  the  task. 

Be  sure  to  include  all  of  the  items  listed  on  Worksheet  07. 


Guidelines  for  Entering  Presentation  Description 

Describing  the  presentation  of  tools,  equipment,  and  materials  is 
fairly  simple.  When  equipment  is  required,  it  can  be  presented  to  the 
performer  in  two  ways: 

1.  Set  up  for  the  performer, 

or 

2.  Performer  is  required  to  set  up. 

For  example,  suppose  the  equipment  is  a  trim  box  (used  in  trimming 
a  jet  engine).  The  trim  box  can  be  presented  to  the  student  already 
cabled  and  ready  to  go  (i.e.,  with  all  the  switches  set  in  their  proper 


position)  or  the  performer  could  be  expected  to  perform  the  cabling  and 
set  the  switches  as  part  of  the  evaluation.  It  is  important  that  all 
performers  receive  the  equipment  in  the  same  manner. 

When  tools  or  materials  are  required,  they  can  be  presented  to  the 
performer  in  two  ways: 

1.  The  performer  must  select, 

or 

2.  Preselected  for  the  performer. 

For  example,  when  tools  are  required,  the  tools  can  be  presented 
in  the  tool  box  or  the  correct  tools  can  be  preselected  and  presented 
to  the  performer. 

For  each  item  you  should  enter  the  appropriate  presentation 
description  in  Block  E  of  Worksheet  08. 

For  equipment  you  should  enter: 

1.  Set  up  for  the  performer, 

or 

2.  Performer  must  set  up. 

For  tools  and  materials  you  should  enter: 

1.  Preselected  for  performer, 

or 

2.  Performer  must  select. 


Block  F  Evaluator  Equipment 


Explanation 

The  evaluator  may  need  some  tools,  equipment,  or  protective 
clothing  in  order  to  conduct  the  evaluation.  These  items  will  be 
listed  in  Block  F  of  Worksheet  08. 


Purpose 


The  information  in  Block  F  tells  the  evaluator  what  tools, 
equipment,  or  protective  clothing  are  needed  to  conduct  the 
evaluation. 


Determining  Evaluator  Equipment 

You  can  determine  the  items  the  evaluator  will  need  by  reviewing 
the  worksheets  already  completed. 

If  time  is  evaluated,  the  evaluator  will  need  a  stopwatch. 

The  evaluator  may  need  tools  to  evaluate  the  end  products  listed 
on  Worksheet  05.  Carefully  examine  the  list  of  end  products.  For  each 
end  product,  ask  yourself,  "Does  the  evaluator  need  any  tools  to 
measure  or  evaluate  the  end  product?" 

The  evaluator  may  also  need  protective  clothing, which  can  usually 
be  determined  by  looking  at  the  protective  clothing  the  performer 
needs.  (These  items  should  be  listed  on  Worksheet  06.)  In  most  cases, 
if  the  performer  needs  ear  plugs,  gloves,  or  aprons,  the  evaluator  will 
also  need  them.  You  should  review  the  protective  clothing  requirements 
for  the  performer  and  decide  which  ones  the  evaluator  will  need. 


Guidelines  for  Entering  Evaluator  Equipment 


Enter  the  evaluator  equipment  in  Block  F.  Some  entries  you  may 

want  to 

consider  are  listed 

below: 

Stopwatch 

Mask 

Gages 

Leather  gloves 

Tape  measure 

Apron 

Dipsticks 

Mat 

Ear  defenders 

Checklists 

Technical  orders 

OJT  manuals 

Block  G 

Evaluator  Time  Estimates 

Explanation 

In 

Block  G,  three  time 

estimates  are  provided: 

1. 

Time  to  set  up  the 

evaluation. 

2. 

Time  to  complete  the  evaluation. 

3. 

Time  to  re-set  up 

the  evaluation. 

This  information  gives  the  evaluator  a  rough  estimate  of  how  much 
time  to  set  aside  for  the  evaluation,  including  setting  up  the 
evaluation  and  the  actual  task  performance. 


Guidelines  for  Entering  Evaluator  Time  Estimates 


Simply  enter  a  rough  time  estimate  on  each  line  in  Block  G. 


Line  H  Number  of  Evaluators 


Explanation 

Some  tasks  may  be  too  complex  to  be  accurately  and  completely 
evaluated  by  one  person.  When  this  happens,  it  is  necessary  to  have 
more  than  one  evaluator. 


Purpose 


This  number  tells  when  more  than  one  evaluator  is  necessary. 


Guidelines  for  Entering  Number  of  Evaluators 


CONSTRUCT  THE  TASK  EVALUATION  FORM 


Explanation 


At  this  point,  you  will  transfer  the  information  you  have  already 
recorded  on  Worksheets  01  to  08  onto  the  TEF.  The  TEF  has  two  parts: 
the  Evaluation  Information  page  and  the  Task  Evaluation  Form  pages. 

The  information  from  Worksheets  02  and  08  will  be  entered  on  the 
Evaluator  Information  page.  The  information  from  Worksheet  01  and 
Worksheets  03  through  07  will  be  entered  on  the  pages  titled  Task 
Evaluation  Form. 

It  will  be  easier  to  complete  Worksheet  09  after  the  information 
from  the  other  worksheets  has  been  entered  on  the  TEF.  Thus,  you 
should  construct  the  TEF  before  you  go  on  to  Worksheet  09  (Scoring) 
Criteria). 

In  this  section,  detailed  instructions  are  provided  for  entering 
information  on  the  Evaluator  Information  page  and  the  Task  Evaluation 
Form  pages. 


Getting  Started 


The  first  thing  you  should  do  is  collect  the  worksheets.  They 
should  be  in  the  order  in  which  they  were  developed. 

This  is  a  good  time  to  review  the  information  you  have  entered  on 
the  worksheets.  Keep  in  mind  that  the  evaluator  will  use  this 
information  to  evaluate  task  performance.  As  you  review  the 
worksheets,  ask  yourself; 

Is  the  information  complete  -  has  every  event  in  the  task  which 
should  be  evaluated  been  included? 

Is  the  information  easy  to  understand  -  can  the  evaluator  use 
the  information  to  decide  whether  or  not  the  task  has  been 
performed  correctly? 

Edit  the  worksheets  as  necessary. 

You  are  now  ready  to  enter  the  information  from  the  worksheets 
onto  the  TEF. 

Examples  of  completed  TEFs  are  contained  in  Appendix  A. 
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Blank  TEFs  can  be  found  in  Appendix  C.  Since  only  10  steps  can  be 
included  on  each  TEF  page,  it  will  be  necessary  to  copy  some  extra 
pages  if  your  task  has  more  than  10  steps.  You  will  probably  want  to 
make  some  extra  copies  of  both  pages  (Evaluator  Information  page  and 
Task  Evaluation  page). 

Before  you  begin  transferring  the  information  from  the  worksheets, 
you  should: 

1.  Enter  the  date  of  development  and  the  developer  (your  name  or 
agency)  on  the  Evaluator  Information  page  in  the  spaces  provided. 

2.  Enter  the  task  title  and  date  of  development  on  each  TEF  page 
in  the  spaces  provided. 

You  are  now  ready  to  begin  with  Worksheet  01. 


Worksheet  01 


The  information  from  Worksheet  01  will  be  entered  in  Column  1  of 
the  page  entitled  Task  Evaluation  Form. 

Enter  the  step  number  from  Column  A  of  the  Worksheet  in  the  box 
inside  of  Column  1  of  the  TEF. 

Enter  the  step  description  from  Column  B  of  the  worksheet  in  the 
large  area  outside  the  small  box  in  Column  1  of  the  TEF. 

The  information  should  be  entered  on  the  TEF  as  follows: 


1.  OesaipTion 

1 

"BTli 

ste 

enl 

The  number  is 
ered  in  the  box,  the 
p  description  is 
ered  here. 

? 

Tnri 

nun 

ior 

Repeat  this  process 
il  all  of  the  step 
ibers  and  descript- 
is  have  been  entered. 

Worksheet  02 


The  information  from  Worksheet  02  will  be  entered  in  Blocks  A  and 
B  of  the  page  titled  Evaluator  Information  page. 

Be  sure  to  enter  the  information  in  the  appropriate  line  on  the 
Evaluator  Information  page. 
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The  information  should  be  entered  as  follows: 

Worksheet  02  Evaluator  Information  Page 


Line 

Line 

Line 

Line 

Line 

Line 


Block 

A 

Block 

A 

Block 

A 

->• 

Block 

A 

Block 

A 

Block 

B 

Worksheet  03 


The  information  from  Worksheet  03  will  be  entered  in  Columns  2  and 
3  of  the  TEF. 

The  steps  of  the  task  are  already  listed  in  Column  1  of  the  TEF. 
Find  the  rows  of  the  TEF  which  correspond  to  the  steps  listed  in  Column 
A  of  the  worksheet. 

The  starting  point  from  Column  B  of  the  worksheet  should  be 
entered  in  Column  2  of  the  TEF.  The  starting  point  should  be  entered 
in  the  row  of  the  TEF  that  corresponds  to  the  step  at  which  the 
evaluator  should  start  measuring  time. 

The  stopping  point  from  Column  C  of  the  worksheet  should  be 
entered  in  Column  2  of  the  TEF.  The  stopping  point  should  be  entered 
in  the  row  of  the  TEF  that  corresponds  to  the  step  at  which  the 
evaluator  should  stop  measuring  time. 

The  standard  from  Column  D  of  the  worksheet  should  be  entered  in 
Column  3  of  the  TEF.  The  standard  should  be  entered  in  the  same  row 
where  the  stopping  point  was  entered. 

Next  you  should  draw  an  arrow  (  |  )  in  Column  2  of  the  TEF.  The 
arrow  should  begin  in  the  row  corresponding  to  the  step  where  you 
entered  the  starting  point.  The  arrow  should  end  in  the  row 
corresponding  to  the  step  where  you  entered  the  stopping  point. 


'U^'t’^TiT^.w^jTT'w^ir^.jr-.x^  w^i»  - 


The  information  from  Worksheet  03  should  be  entered  as  follows: 


2.  Surt^top 
Miftsuring 

3.  Standard 

The  starting  point 
is  entered  here 
across  from  the 
appropriate  step.- 

The  stopping  point 
is  entered  here 
across  from  the 
appropriate  step. 

The  time 
a1 1  owed 
is  enter¬ 
ed  here. 

Worksheet  04 


The  information  from  Worksheet  04  will  be  entered  in  Column  4  of 
the  TEF. 

In  Block  A  of  the  worksheet,  you  entered  the  series  of  steps  which 
should  be  evaluated.  Find  the  row  of  the  TEF  which  corresponds  to  the 
first  step  of  the  series.  Enter  the  statement  from  Block  A  of  the 
worksheet  in  the  row  of  the  TEF  that  corresponds  to  the  first  step  of 
the  series. 

Next,  draw  an  arrow  in  Column  4  of  the  TEF  from  the  first  step  in 
the  series  to  the  last  step  in  the  series. 

Repeat  this  process  for  each  series. 

In  Block  B  of  the  worksheet,  you  entered  single  steps  which  should 
be  performed  before  other  steps.  Find  the  row  of  the  TEF  which 
corresponds  to  the  first  step.  Enter  the  statement  from  Block  B  of  the 
worksheet  in  the  row  of  the  TEF  that  corresponds  to  the  first  step. 

Repeat  this  process  for  each  single  step. 
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The  information  from  Worksheet  04  should  be  entered  on  the  TEF  as 
fol lows: 


Worksheet  05 


The  information  from  Worksheet  05  will  be  entered  in  Column  5  of 
the  TEF. 

For  each  end  product,  find  the  row  of  the  TEF  that  corresponds  to 
the  step  listed  in  Column  C  of  the  worksheet. 

Enter  the  end  product  from  Column  A  and  the  criteria  from  Column  B 
in  the  row  of  the  TEF  which  corresponds  to  the  step  listed  in  Column  C. 
If  an  end  product  is  preceded  by  a  check  (.,  ),  be  sure  to  enter  the 
check  on  the  TEF.  ' 


When  more  than  one  type  of  criteria  are  listed  for  an  end  product, 
enter  a  slash  mark  (/)  after  each  criterion  type.  You  do  not  have  to 
re-enter  the  end  product  for  each  criterion  type. 

The  information  from  Worksheet  05  should  be  entered  on  the  TEF  as 
follows: 


S.  End-Product  Cfiterii 

End  product  criterion  type  1/ 
criterion  type  2/  criterion  type 
3/ 


\/  End  Product  with  one  criterion 
type  and  check  mark. 


Column  b  completed  in  correct  row/ 
criteria  separated  by  slashes/ 


Criteria  types  are  separated  by 
slash  marks;  but  an  end  product 
does  not  have  to  be  re-entered  for 
each  criterion  type. 


Worksheet  06 


The  information  from  Worksheet  06  will  be  entered  in  Column  6  of 
the  TEF. 


For  each  safety  procedure  or  regulation,  find  the  row  of  the  TEF 
which  corresponds  to  the  steps  listed  in  Column  B  of  the  worksheet. 
Enter  the  descriptions  of  the  safety  procedures  and  regulations  in  the 
row  of  the  TEF  which  corresponds  to  the  step  listed  on  the  worksheet. 

When  a  particular  safety  procedure  or  regulation  applies  to  more 
than  one  step,  repeat  the  entry  for  every  applicable  step. 

When  there  is  more  than  one  entry  for  a  step,  enter  a  slash  (/) 
after  each  entry. 


The  information  from  Worksheet  06  should  be  entered  on  the  TEF  as 
fol lows; 


6.  Procedures  and  Regulations 

The  safety  procedure  or  regulation 
is  entered  in  the  correct  row. 


The  safety  procedure  or  regulation 
is  entered  in  the  correct  row/When 
more  than  one  safety  procedure  or 
regulation  is  entered  for  a  step, 


The  safety  procedure  or  regulation 
is  entered  in  the  correct  row/The 
safety  procedure  or  regulation  is 
entered  at  every  applicable  step/ 


NOTE: 

When  there  is  more  than  one  safety 
procedure  or  regulation  for  a  step, 
each  entry  is  followed  by  a  slash 
mark. 


Worksheet  07 


The  information  from  Worksheet  07  will  be  entered  in  Column  7  of 
the  TEF. 

For  each  item,  find  the  row  of  the  TEF  which  corresponds  to  the 
step  listed  in  Column  D  of  the  worksheet.  Enter  the  information  from 
Columns  A,  B,  and  C  of  the  worksheet  in  the  row  of  the  TEF  that 
corresponds  to  the  step  listed  on  the  worksheet. 

When  a  particular  item  is  used  at  more  than  one  step,  repeat  the 
entry  for  every  applicable  step. 

When  there  is  more  than  one  entry  for  a  step,  enter  a  slash  mark 
(/)  after  each  entry. 
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The  information  from  Worksheet  07  should  be  entered  on  the  TEF  as 
fol lows: 


7.  Item,  Site  or  Type,  Use 


The  item,  size  or  type,  and  use  are 
entered  in  the  correct  row. 


The  item,  size  or  type,  and  use  are 
entered  at  every  applicable  step. 

Item  1,  size  or  type,  use/ 

Item  2,  size  or  type,  use/ 

Item  3,  size  or  type,  use/ 


NOTE: 

When  there  is  more  than  one  item  for 
a  step,  each  entry  is  followed  by  a 
slash  mark. 


Worksheet  08 


The  information  from  Worksheet  08  will  be  entered  on  the  page 
titled  Evaluator  Information. 

Be  sure  to  enter  the  information  from  Worksheet  08  onto  the 
appropriate  Block  of  the  Evaluator  Information  page. 

The  information  from  Worksheet  08  should  be  entered  on  the  TEF  as 
follows: 


Worksheet  08 

Line  A  — 
Block  B  — 
Block  C  — 
Block  D  — 
Block  E  - 
Block  F  _ 
Block  G  - 
Block  H  — 


Evaluator  Information  Page 


Block  C 
Block  0 
Block  E 
Block  H 
Block  F 
Block  G 
Block  I 
Block  J 


You  have  finished  entering  all  the  information  from  Worksheets  01 
through  08  onto  the  TEF. 

Number  the  pages  of  the  TEF  in  the  spaces  provided. 


ila  ji_«  ii_^  j»  A  •  \  .  .  . 
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Automatic  Failure 


Explanation 


There  is  one  more  thing  you  should  do  to  complete  the  construction 
of  the  TEF.  You  have  already  entered  the  parts  of  the  task  that  should 
be  evaluated  for  each  evaluation  area.  Now  you  will  identify  those 
entries  that  could  involve  the  most  serious  consequences  if  not 
correctly  performed.  Often  there  are  a  few  entries  on  the  TEF  that,  if 
not  performed  correctly,  would  result  in  automatic  failure  and  possibly 
termination  of  the  evaluation.  This  includes  those  entries  that,  if 
not  performed  correctly,  result  in  failure  of  the  evaluation  —  even  if 
all  of  the  other  entries  are  correctly  performed. 


Purpose 


The  entries  that  could  result  in  automatic  failure  and  possibly 
termination  of  the  evaluation  will  be  marked  on  the  TEF  for  the 
evaluator.  During  the  evaluation,  if  one  of  these  entries  is  not 
correctly  performed,  the  evaluator  will  automatically  fail  the 
performer  and  possibly  terminate  the  evaluation. 


Finding  Automatic  Failures 

You  should  review  all  of  the  entries  in  each  evaluation  area.  Ask 
yourself: 

1.  Will  successful  completion  of  the  task  be  prevented  even  if 
everything  else  is  performed  correctly? 

2.  Will  such  damage  to  equipment  result  that  the  evaluation  would 
be  terminated? 

3.  Will  such  a  risk  of  injury  result  that  the  evaluation  would  be 
terminated? 

Some  tasks  will  not  have  entries  that  lead  to  automatic  failure. 
The  most  common  automatic  failures  are  extreme  safety  hazards  or 
important  end  product  criteria. 

REMEMBER,  ONLY  THOSE  ENTRIES  THAT  COULD  RESULT  IN  AUTOMATIC 
FAILURE— EVEN  WHEN  EVERY  OTHER  ENTRY  ON  THE  TEF  IS  PERFORMED 
CORRECTLY— SHOULD  BE  CONSIDERED  HERE. 


Guidelines  for  Entering  Automatic  Failures 


The  entries  which  could  lead  to  automatic  failure  should  be  marked 
with  an  asterisk  (*)  on  the  TEF. 

The  instructions  below  describe  exactly  where  asterisks  should  be 
placed  in  each  evaluation  area. 

Time/Speed.  Place  the  asterisk,  if  appropriate,  before  the 
standard  in  Column  3. 

Sequence-Fol lowing.  Rarely  will  every  step  in  a  sequence  qualify 
as  automatic  failure.  There  are  times  when  several  steps  in  a  long 
sequence  will  qualify.  These  steps  should  be  identified  with  an 
asterisk  in  Column  4  in  the  upper  left-hand  corner. 

End-Product.  Place  the  asterisk,  if  appropriate,  before  the 
criteria  which  qualify.  Do  not  place  the  asterisk  before  the 
end  product. 

Safety.  Place  the  asterisk,  if  appropriate,  before  the  procedure 
or  regul ati on  which  qualifies.  When  a  procedure  or  regulation  could 
result  in  automatic  failure  at  more  than  one  step,  place  an  asterisk 
before  each  entry  which  qualifies. 

Tools,  Equipment,  and  Material  Use.  Place  the  asterisk,  if 
appropriate,  before  the  tools,  equipment,  or  material  which  qualifies. 
When  an  item  could  result  in  automatic  failure  at  more  than  one  step, 
place  an  asterisk  before  each  entry  which  qualifies. 

REMEMBER,  ASTERISKED  ENTRIES  WILL  RESULT  IN  AUTOMATIC  FAILURE, 

EVEN  IF  EVERY  OTHER  EVENT  ON  THE  TEF  IS  CORRECTLY  PERFORMED. 

Enter  the  asterisks  to  the  left  of  the  TEF  entries  that  could 
result  in  automatic  failure. 


Examples  of  Automatic  Failure 

For  examples  of  automatic  failure,  see  the  completed  TEF  for  the 
LOX  servicing  task  in  Appendix  A.  (The  preflight  task  does  not  have 
any  automatic  failures.) 


YOU  ARE  NOW  READY  TO  GO  ON  TO  WORKSHEET  09. 


WORKSHEET  09:  SCORING  CRITERIA 


Overview 


Explanation 


Worksheet  09  is  used  to  assign  points  to  each  non-asteri sked  entry 
on  the  TEF.  The  number  of  points  assigned  to  each  entry  in  an 
evaluation  area  is  determined  by  three  factors  as  follows: 

1.  The  number  of  points  assigned  to  the  evaluation  area. 

2.  The  number  of  total  entries  in  the  evaluation  area. 

3.  The  number  of  asterisked  entries  in  the  evaluation  area. 

A  formula  for  calculating  the  points  per  non-asteri sked  entry  is 
included  on  Worksheet  09.  Completing  the  formula  involves  the 
following  steps: 

1.  The  evaluation  areas  are  ranked  in  order  of  their  importance 
in  the  task. 

2.  The  points  assigned  to  each  evaluation  area  are  based  on  the 
assigned  ranks. 

3.  The  number  of  asterisked  entries  and  the  number  of  total 
entries  per  evaluation  area  are  counted. 

4.  The  points  per  evaluation  area  are  divided  among  the  entries 
in  the  evaluation  area.  Twice  as  many  points  are  assigned  to 
asterisked  entries. 

5.  The  formula  results  in  the  number  of  points  per  non-asterisked 
entries. 


Purpose 

The  number  of  points  per  non-asterisked  entry  is  transferred  to 
the  Evaluator  Information  page.  This  tells  the  evaluator  the  number  of 
points  that  should  be  subtracted  for  each  non-asterisked  entry  which  is 
not  successfully  completed.  It  is  important  that  the  number  of  points 
subtracted  is  standardized;  i.e.,  each  evaluator  subtracts  the  same 
number  of  points  for  the  same  error. 

The  points  per  asterisked  entry  are  not  entered  on  the  Evaluation 
Information  page.  When  an  asterisked  entry  is  not  successfully 
completed,  automatic  failure  results.  A  score  of  "0"  is  assigned  when 
an  automatic  failure  occurs. 


Completing  Worksheet  09 

In  order  to  complete  Worksheet  09  and  derive  the  number  of  points 
per  non-asterisked  entries,  you  will: 

1.  Rank  order  the  evaluation  areas. 

2.  Count  the  total  entries  in  each  evaluation  area. 

3.  Count  the  asterisked  entries  in  each  evaluation  area. 

4.  Find  the  number  of  points  per  evaluation  area  by  using 
Appendix  D. 

5.  Calculate  the  points  per  non-asterisked  entry  by  using  the 
formula  on  Worksheet  09. 

Instructions  are  provided  for  completing  each  box  of  Worksheet 
09. 


Examples  of  Worksheet  09 

Examples  of  Worksheet  09  are  shown  in  Figures  17  and  18. 


Figure  17.  Example  of  Completed  Worksheet  09: 
LOX  Servicing  8-52H. 
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orksheetOS:  Scoring  Criteria 


Block  A  Rank 


Explanation 

Often,  one  or  more  of  the  evaluation  areas  are  more  important  to 
task  performance  than  others.  At  this  point,  you  will  rank  the 
evaluation  areas  in  order  of  their  importance  to  successful  task 
performance. 


Purpose 

One  hundred  points  are  assigned  to  the  whole  task.  This  total 
number  of  points  is  divided  among  the  five  evaluation  areas. 
Evaluation  areas  that  you  rank  higher  than  others  will  receive  more 
points,  and  evaluation  areas  that  are  ranked  the  least  important  will 
be  worth  fewer  points. 

Appendix  0  contains  a  chart  for  assigning  points  to  each 
evaluation  area  based  on  the  combination  of  ranks  you  assign  to  the 
task. 


Assigning  Ranks 

The  evaluation  areas  are  ranked  from  the  least  important  (1)  to 
the  most  important  (the  highest  rank).  You  can  assign  any  combi natio 
of  ranks  you  think  is  appropriate.  Two  or  more  evaluation  areas  can 
have  the  same  rank  if  they  are  equal  in  importance. 

You  should  follow  the  guidelines  below: 

1.  Assign  the  least  important  evaluation  area(s)  the  rank  of  1. 

2.  Assign  the  most  important  evaluation  area(s)  the  highest 

rank. 


3,  Do  not  skip  numbers. 

4.  Do  not  assign  a  rank  when  a  evaluation  area  has  no  entries. 


Guidelines  for  Entering  Ranks 

Enter  the  rank  for  each  evaluation  area  in  Block  A  of  Worksheet 

og. 


Enter  N/A  in  Block  A  if  there  are  no  entries  for  an  evaluation 


area. 


Some  examples  of  combinations  of  ranks  are: 


Evaluation  Areas 

T ime/Speed 
Sequence-Fol lowing 
End  Product 
Safety 
TEM  Use 


Ranks 

N/A  5  1  1 

3  2  2  N/A 

2  13  1 

3  3  2  2 

1  4  2  2 


Block  B  All  Entries 


Explanation 


The  number  of  entries  in  each  evaluation  area  of  the  TEF  is  the 
first  number  needed  to  complete  the  formula  on  Worksheet  09.  The  total 
number  of  entries  should  include  both  asterisked  and  non-asterisked 
entries. 


Counting  Entries 

Guidelines  for  counting  the  entries  in  each  evaluation  area  are 
provided  below: 

1.  Time/Speed  -  Each  separate  segment  where  time  is  evaluated  is 
considered  one  entry. 

2.  Sequence-Following  -  If  series  are  included  on  the  TEF,  each 
step  in  a  series  is  considered  one  entry.  For  example,  if  a  TEF 
included  the  entry  Steps  2  through  10,  nine  entries  would  be  counted. 

If  single  steps  are  included  on  the  TEF,  each  last  step  is  considered 
one  entry.  For  eximple,  if  a  TEF  included  Step  3  before  Steps  9,  10, 
and  11,  three  eni.  -  would  be  counted. 

3.  End  Product  -  Each  type  of  criteria  listed  is  considered  one 
entry.  For  example,  if  two  end  products  were  included  on  the  TEF,  with 
the  first  having  one  type  of  criteria  and  the  second  having  three  types 
of  criteria,  four  entries  would  be  counted. 

4.  Safety  -  Each  procedure  or  regulation  included  on  the  TEF 
counts  as  one  entry.  When  a  procedure  or  regulation  is  repeated  for 
more  than  one  step,  it  is  counted  as  an  entry  every  time  it  appears. 


-V, 

.V, 


5.  TEM  Use  -  Each  tool,  equipment  item,  or  material  with  its 
specific  use,  counts  as  one  entry.  When  an  item  is  repeated  for  more 
than  one  step,  it  is  counted  as  an  entry  every  time  it  appears. 


Guidelines  for  Entering  All  Entries 


Enter  the  number  of  entries  for  each  evaluation  area  in  Block  B  of 
Worksheet  09. 


Block  C  Asterisked  Entries 


Explanation 


The  next  number  you  need  to  complete  the  formula  on  Worksheet  09 
is  the  number  of  asterisked  entries  in  each  evaluation  area. 


Guidelines  for  Entering  Asterisked  Entries 


Simply  enter  the  number  of  asterisked  entries  for  each  evaluation 
area  in  Block  C  of  Worksheet  09. 


Block  0  Points  Per  Evaluation  Area 


Explanation 

A  number  of  points  will  be  assigned  to  each  evaluation  area  based 
on  the  ranks  you  already  entered  in  Block  A.  These  points  will  be 
divided  among  the  entries  in  each  evaluation  area  to  come  up  with  the 
number  of  points  per  non-asteri sked  entries. 


Assigning  Points 


You  should  use  the  chart  in  Appendix  0  to  convert  the  ranks  into 
points  per  evaluation  area.  Simply  follow  the  flowchart  until  you  have 
found  the  point  value  for  each  rank  you  assigned. 


Guidelines  for  Entering  Points  Per  Evaluation  Area 


Enter  the  points  for  each  evaluation  area  in  Block  D  of  Worksheet 


100 


Block  E  Total  B  +  C 


Explanation 


Now  you  have  all  the  numbers  necessary  to  complete  the  formula  on 
Worksheet  09.  First  you  should  add  Block  B  (All  Entries)  and  Block  C 
(Asterisked  Entries).  Repeat  this  for  each  evaluation  area. 


Guidelines  for  Entering  Total  B  -t-  C 

Enter  the  totals  for  Blocks  B  +  C  in  Block  E  of  Worksheet  09  for 
each  evaluation  area. 


Block  F  Points  Per  Non-Asterisked  Entries 


Explanation 


You  are  now  ready  to  find  the  points  per  non-asterisked  entry. 

For  each  evaluation  area,  simply  divide  Block  D  (Points  for  evaluation 
area)  by  Block  E  (Total  B  +  C). 


Guidelines  for  Entering  Points  Per  Non-Aster isked  Entry 

Enter  the  results  of  dividing  Block  D  by  Block  E  in  Block  F. 
YOU  ARE  NOW  READY  TO  COMPLETE  THE  EVALUATOR  INFORMATION  PAGE. 


COMPLETING  THE  EVALUATOR  INFORMATION  PAGE 


Block  K  Scoring  Criteria 


Explanation 


The  last  step  in  TEF  development  is  to  transfer  the  information 
from  Blocks  D  and  F  of  Worksheet  09  onto  the  Evaluator  Information 
page.  The  information  will  be  transferred  to  Block  K,  Scoring  Criteria 
of  the  Evaluation  Information  page. 


Purpose 

When  the  points  per  non-asteri sked  events  are  included  on  the 
Evaluator  Information  page,  the  evaluator  can  use  this  information  to 
score  the  evaluation. 


Transferring  the  Scoring  Criteria 

Enter  the  points  per  each  evaluation  area  from  Block  D  of 
Worksheet  09  in  the  column  labeled  "Total  Points  Possible"  in  Block  K 
of  the  Evaluator  Information  page. 

Enter  the  number  of  points  from  Block  F  of  Worksheet  09  in  the 
column  labeled  "Points"  in  Block  K  of  the  Evaluator  Information  page. 

When  an  evaluation  area  has  no  entries,  enter  N/A  in  each  column 
on  the  Evaluator  Information  page. 


Building  Search 
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Examples  of  completed  Forms  for  two  tasks,  LOX  Servicing  B-52H  and 
Preflight  Inspection  KC-135A,  are  included  in  this  Appendix.  You  will 
notice  that  there  are  two  different  types  of  pages.  The  first  page  of 
each  example  is  titled  Evaluator  Information.  The  remaining  pages  of 
each  example  are  titled  Task  Evaluation  Form. 

The  Evaluation  Information  page  is  used  by  the  evaluator  to  set-up 
the  evaluation  and  to  score  observed  performance  after  the  evaluation 
is  complete.  The  Task  Evaluation  Form  pages  are  used  during  task 
performance  to  guide  the  evaluator's  observations  and  to  record  errors. 
Specific  information  about  the  two  forms  is  provided  below. 


Evaluator  Information  Page 


Blocks  A  through  J  provide  information  to  the  evaluator  about  the 
task  to  be  evaluated  and  how  the  task  should  be  presented  to  the 
performer  for  evaluation  purposes. 

Block  K  includes  a  scoring  chart  which  the  evaluator  completes 
after  the  task  performance. 


Task  Evaluation  Form  Pages 


Column  1  lists  the  steps  of  the  task  which  will  be  performed  for 
the  evaluation. 

Columns  2  through  7  describe  what  should  be  evaluated  at  each  step 
of  the  task.  The  evaluator  uses  this  information  as  standards  of  task 
performance.  The  evaluator  circles  the  correspond ing  entries  on  the 
form  when  errors  occur. 
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EVALUATOR  INFORMATION 

A.  TASK  DESCRIPTION 

DATE  OF  DEVELOPMENT;  6/85  DEVELOPER: 

AFSC/OUTY  POSITION;  431X2 
TASK  TITLE:  LOX  Strvlclng  a-52H 

TASK  BEGINNING:  Aa  p«rforMr  poaittoaR  fir*  •xcUkiuiah«r 
TASK  END:  Aft«r  Rxtra  grounding  cnblo  in  diRconnactod 

STEPS  OR  EVENTS  NOT  INCLUDED  IN  THE  EVALUATION:  Stop  1.  Stop  2,  annotating  AFTO  fora 


B.  TASK  INFORMATION  SOURCES 

TITLE  DATE 

Technical  OrdarR  lg-S2H-2-2JO-t.  3  10/82 
OJT  Instructor 'r  CuldR  No*  8506  11/82 


C.  EVALUATION  METHOD 

Actual  BqulpBRnt  Job  Envlronaant 

D.  PREVENTATIVE  ENVIRONMENTAL  CONDITIONS 
ThundarRtorBR  irlthln  3  nIIrr 


E.  PRESENTATION  OF  OPERATIONAL  EQUIPMENT/TASK 
Aircraft  raqulraa  LOX  aarvlclng 
Aircraft  grounded 

Aircraft  not  within  50  ft.  of  other  aircraft 
No  flervlcing  of  fuel*  oil,  hydraullcR*  or  water 
electrical  power  off 
Paraonnal  off  aircraft 

No  oxygen  leahSt  no  coaponenca  ratBovadi  no  purging 
required 


K.  SCORING  CRITERIA 


NOTE  TO  EVALUATOR: 
It  ii  Imoonem  thti  *11 
•vafueiori  tcort  the  TEF 
in  the  am«  wty.  If  you 
hcv*  never  scored  e  TEF 
Of  ere  imiure  ebout  the 
number  of  errori  to  enter. 
pfetM  tec  the  TEF  EVAL 
UATOR  INSTRUCTIONS. 


*Cifde  fall  wfih  e  teera 
of  tare  II  any  ettarltlicd 
errori  Recurred. 


F.  PRESENTATION  OF  PERFORMER  TOOLS. 
EQUIPMENT.  AND  MATERIAL 

TOOLS.  EQUIPMENT.  MATERIAL 

Hlniklt  i 

Rag*  p 

Fece  Shield 

Leather  Glovaa  with  Wool  Inaarta 
Aprona 

Hat  (non*aaah) 

Slaavaa 

Extra  Ground  Wire 
LOX  Cart 
Drip  Pena 
Fire  Bottle 


C%.  CVAI.UAIOF  TOOLS.  E0U»PVBUT.  AND  MATERiAL 

Stopwatch 
Face  Shield 

Leather  Glovaa  with  Wool  Inaarta 
Aprona 

Hat  (non-aaah) 

Slaavaa 


I.  EVALUATOR  TIME  ESTIMATES 

Tim.  to  S«Up; _ 

Tim.  to  E..IU.I.;  alni.r.. 
Time  to  Retat  N/A__ 


J.  NUMBER  OF  EVALUATORS 


PRESENTATION 

All  praaalactad  for  the 
parforaar 


IVAIUATION 

AMA 

1 

TOTAt 

acuiTS 

potsiact 

huiiafa 

AdTCnCKCO 

laeoas* 

TIhw^PMC 

9 

faOomnf 

27  . 

twcarxMcr 
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TASK  EVALUATION  FORM 


A.  TASK  OESCniPTiON 


EVALUATOR  INFORMATION 


DATE  OF  DEVELOPMENT:  g/gS  DEVELOPER: 

AFSC/OUTY  POSITION:  43iX2 

TASK  TITLE:!  Pr«£llght  KC-135A 

task  BEGINNING:  Beginning  o£  Work  Card  1-003 

TASKENO:  End  of  Work  Card  1-0 10 


STEPS  OR  EVENTS  NOT  INCLUDED  IN  THE  EVALUATION;  Work  Card  08  dlacrepancles  found  and  servicing  necessary  will  only 
be  annotated  (l.e*.  LOX  Servicing,  Fire  Extinguisher  Servicing) 


B.  TASK  INFORMATION  SOURCES 

TITLE  DATE 

Technical  Orders  K-135(K)  A-2-9J6-3  3/82 

OJT  Instructor's  Guide  No.  8101  1/82 


E.  PRESENTATION  OF  OPERATIONAL  EOUlPMENT/TASK 

No  other  aalntenance  being  perfomsd 
Aircraft  requires  prefllght  Inspection 
Aircraft  grounded 


H.  HELP  PERMITTED 


K.  SCORING  CRITERIA 


c.  Evaluation  method 

Actual  Equlpiaent  Job  Envlronoient 


0.  PREVENTATIVE  ENVIRONMENTAL  CONDITIONS 
N/A 


F.  presentation  OF  PERFORMER  TOOLS. 
EOUIPMENT,  AND  MATERIAL 


TOOLS.  EQUIPMENT.  MATERIAL 

Hlnlklt 
Tire  Cage 
Tape  Measure 
Work  Cards 
Rags 

B-l»  B-4,  B-5  stands 


presentation 

Preselected 
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Preselected 

Preselected 

Preselected 


G.  EVALUATOR  TOOLS.  EQUIPMENT.  AND  MATERIAL 


Tire  Cage 
Work  Cards 


1.  EVALUATOR  TIME  ESTIMATES 
Tims  to  Set  up:  MVA. 


Tima  to  Evaluaia:  alnutee 

Tima  to  Ratal  -  H/A 
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APPENDIX  B 
BLANK  WORKSHEETS 


I 


I 
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Worksheet  01;  Task  Steps 


Task : 


Developer: 


Column  A:  # 


Column  B;  Step  Description 


Worksheet  03:  Evaluation  of  Time  or  Speed  of  Task  Performance 


Worksheet  04:  Evaluation  of  Sequence-Fol lowing 

Task: 

Developer: 

Block  A 

Series  of  Steps 

Step _ 

through 

Step _ 

through 

Step _ 

through 

Step _ 

through 

Step _ 

through 

Block  B 

Single  Steps 

Step 

before  Step(s) 

Step 

before  Step(s) 

Step 

before  Step(s) 

Step _ 

before  Step(s) 

step _ before  Step(s) 


Worksheet  05:  Evaluation  of  End  Product 


Worksheet  06:  Evaluation  of  Safety  Procedures  and  Regulations 


Worksheet  07:  Evaluation  of  Tools,  Equipment,  and  Materials  Use 


EVALUATOR  INFORMATION 


A.  TASK  OCSCFIFTION 

DATE  OF  DCVELOFMENT: 

AFSC/OUTY  POSITION; 

TASK  TITLE:! 

TASK  tEGINNING: 

TASK  END: 

STEPS  OR  EVENTS  NOT  INCLUOED  IN  THE  EVALUATION: 


B.  TASK  INFORMATION  SOURCES 

TITLE 

OATS 

D.  PREVENTATIVE  ENVIRONMENTAL  CONDITIONS 


I.  EVALUATOR  TIME  ESTIMATES 
Tlm«  to  Sot'Up: 

TIim  to  Evtiuat*:  — 
Tbnt  to  R«Mt  - 


i.  NUMBER  OF  EVALUATORS 


K.  SCORING  CRITERIA 
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APPENDIX  0 

CHART  FOR  DETERMINING  POINTS  PER  EVALUATION  AREA 
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Many  Evatuatton 
Areas  Are  Assigned 
The  Lowest  Hank. 
^^(Hank  1) 


2,  3,  4  or  5 


^  How  V 
Many  Evaluation 
Areas  Are  Assigned 
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^  Many  Evaluation 

Many  Evaluation 
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Rank  1  =  10 
Rank  2  =  20 
Rank  3  =  30 


Rank  1  =  11 
Rank  2  =  22 
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Rank  4 
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